nixfredFLY.IOFIELD GUIDE · BY NIX UNOFFICIAL
EXECUTE / RELIABILITY, HONESTLY

Reliability, honestly

What Fly.io says about its own architecture risks and incidents, what a host failure means for an app with one Machine versus two, why volumes pin an app to a host, what the docs recommend for high availability, and how to talk to a customer who just had an incident. Every incident below links to Fly’s own write-up or to the live status history. Nothing here is an uptime guarantee; the only SLA in the sources is on the Enterprise support plan.

This page is drawn from https://status.flyio.net/history (fetched 2026-11-04), https://fly.io/infra-log, https://fly.io/blog/corrosion/, https://fly.io/blog/parking-lot-ffffffffffffffff/, https://fly.io/docs/apps/trouble-host-unavailable/, https://fly.io/docs/blueprints/resilient-apps-multiple-machines/, https://fly.io/docs/volumes/overview/ and community.fly.io threads.

What the status page shows

The status history page for the window it displayed, 2 Oct 2026 to 4 Nov 2026, lists 26 entries (derived: my count of the entries on https://status.flyio.net/history, fetched 2026-11-04). They mix incidents and planned work, so do not read 26 as 26 outages. The shape is more useful than the count:

  • Mostly regional, mostly short. Examples as listed: “Emergency network maintenance on a subset of hosts in ORD” (4 Nov), “Elevated Machine start failures in ARN” (2 Nov), “Outbound IPv4 connectivity failures in ARN, JNB, and GRU” (1 Nov), “Slow image pulls delaying Machine starts in IAD” (30 Oct), “Capacity issues in GRU region” (20 Oct), “Partial FRA network issues” (14 Oct, “around 10 minutes”) (https://status.flyio.net/history).
  • Control-plane and tooling events appear too. “Elevated issues with the Machines API” (29 Oct), “Elevated errors creating apps and managing secrets” (15 Oct, with the note “Running apps and Machines were not affected”), and an upstream builder incident listed as “Elevated Depot Failures” (29 Oct) (https://status.flyio.net/history).
  • An entry titled platform-wide: “Platform Wide Rolling Host Reboots” (17 Oct), closed with “A small number of hosts remain degraded or offline for apps and for MPG”, and a pointer to the personalized status page at https://fly.io/status (https://status.flyio.net/history).
  • Sprites had its own events: “Sprite hosting outage” (listed 11 Oct, “some or all hosted sprites on the *.sprites.app subdomain were unavailable due to a certificate error”) (https://status.flyio.net/history).

Two limits to say out loud. The public page is not the whole picture: individual host issues are not posted there, only on the affected customer’s dashboard (https://fly.io/docs/apps/trouble-host-unavailable/). And the history page shows only the window the page displays; I did not fetch older months, so claims about trend over a year are not supported here [verify].

Fly’s own incident log

Fly publishes an Infra Log and states its scope: “incidents are internal events for our infrastructure and engineering teams”, and it “aims for 100% fidelity to internal incidents” as a superset of status-page events (https://fly.io/infra-log). Entries I read on 2026-11-04:

DateWhat happenedFly’s stated cause and follow-upSource
24 Sep 2026Routing in GRU degraded for “approximately two and a half hours”; “Other regions were unaffected, and all existing Machines continued running throughout the incident.”A rollout of a new replication metadata format for Corrosion (Fly’s state-propagation system) to the GRU cluster hit a bug; recovery was prolonged by a mismatched peer address. “Our existing testing did not catch this issue before it reached production.”https://fly.io/infra-log
8 Oct 2026Loss of inbound connectivity through LHR edges for “roughly 15 minutes”.“we decommissioned the old hosts before confirming the new ones were correctly advertising routes over BGP”; the health check meant to catch this passed anyway and is being tightened.https://fly.io/infra-log
12 Oct 2026Hosted Sprites unavailable for “roughly 75 minutes”.A certificate renewal bug met a proxy fallback bug; “We lacked automated alerting for Sprite URLs”. Proxy fixed since.https://fly.io/infra-log
15 Oct 2026Secrets store write failures: “a little over half of write requests started failing”.Added write volume caused SQLite lock contention; Fly shipped a version without the tracking code.https://fly.io/infra-log

Note the tone: Fly names its own failed checks and missing alerts. That candour is useful in a customer conversation, because you can point to it instead of defending the platform.

Architecture risks Fly has written about

  • One global routing state. On 1 September 2024 a new Machine with a new “virtual service” option made every proxy lock up (the post says they “locked up hard”), “a period during which no end-user requests could reach our customer apps at all” (https://fly.io/blog/corrosion/). The post’s own framing: “Distributed systems are blast amplifiers.” (https://fly.io/blog/corrosion/)
  • The fix is a watchdog plus regionalization. Fly added watchdogs (“Watchdogs have cancelled multiple outages”) and says “we built a single global state domain” and is unwinding it (https://fly.io/blog/corrosion/). An earlier post explains why this is slow: “every proxy received updates for every Fly Machine”, and at the time of that post (28 May 2025) that was still true (https://fly.io/blog/parking-lot-ffffffffffffffff/).
  • More Corrosion trouble. The same post describes a schema change that made the cluster melt down and an expired Consul certificate whose retries saturated uplinks: “We apologize to our uplink providers.” (https://fly.io/blog/corrosion/)

Advice: when a customer asks “could the whole platform go down at once?”, the honest answer from Fly’s own writing is that it has happened (the 2024 proxy deadlock), that Fly says it is reducing the blast radius, and that redundancy inside one app does not protect against a platform-wide event. Multi-provider design is the customer’s call; the sources do not give Fly guidance on it [verify].

One Machine versus two

flowchart LR
  subgraph One[One Machine]
    A1[Host fails] --> A2[App down until host returns<br/>or you rebuild elsewhere]
  end
  subgraph Two[Two Machines on separate hosts]
    B1[One host fails] --> B2[Fly Proxy routes to the other]
  end
Whiteboard diagram (rendered from the source mermaid)

Volumes pin you to a host

What the docs recommend for high availability

  1. At least two Machines per process, in more than one region where it matters (https://fly.io/docs/blueprints/resilient-apps-multiple-machines/, https://fly.io/docs/getting-started/troubleshooting/).
  2. Autostop/autostart or standby Machines to keep the second Machine cheap (https://fly.io/docs/blueprints/resilient-apps-multiple-machines/).
  3. Health checks so the proxy stops routing to a bad Machine (https://fly.io/docs/apps/app-availability/).
  4. Your own session storage, replication and backups: the platform does not do these for volumes (https://fly.io/docs/apps/app-availability/).
  5. For Postgres, Managed Postgres advertises “High availability with automatic failover” (https://fly.io/docs/postgres/). Fly’s older unmanaged Postgres is described by Fly staff in 2022 as “not a managed DB”, and by the Fly blog as “not a fully managed database” (https://community.fly.io/t/anatomy-of-a-postgres-outage/8167, https://fly.io/blog/logbook-october-21-to-28-2022/).
  6. Single-region caveat: two Machines in one region survive a host failure, not a regional one. Spread with fly scale count 2 --region iad,ord (https://fly.io/docs/getting-started/troubleshooting/).

Talking to a customer after an incident

Say:

Never say:

  • This will not happen again. Fly’s own entries show repeat classes of failure (certificates, routing state, edge changes) and say what is being tightened, not that failure is over (https://fly.io/infra-log).
  • “It was your config” before you have checked the host. Host issues are absent from the public page (https://fly.io/docs/apps/trouble-host-unavailable/).
  • Any uptime figure that is not the Enterprise 99.9% SLA, and then only for customers on that plan (https://fly.io/support).
  • Anything about root cause that Fly has not published.

When Fly was wrong, borrow its tone. On a 2022 docs mistake a Fly staffer wrote that Fly had “let you down here with our docs” and apologized (https://community.fly.io/t/anatomy-of-a-postgres-outage/8167): own the failure, state the fix, move on.

← Support playbook · Economics →