LEARN / THE ARCHITECTURE

The Fly.io architecture

Fly.io has four moving parts: edge servers that take traffic, worker servers that run Firecracker microVMs, a WireGuard mesh that joins them, and a gossiped state catalog called Corrosion that tells the proxies where everything is. Once you hold those four, you can explain most product behavior.

This page is drawn from https://fly.io/docs/reference/architecture/, https://fly.io/docs/reference/fly-proxy/, https://fly.io/docs/security/shared-responsibility/ and https://fly.io/docs/machines/overview/, plus the engineering posts on https://fly.io/blog/ cited inline. Several posts are years old. Where a post may be stale, the text says so.

The request path, end to end

flowchart LR
  U[User] -->|DNS gives Anycast IP| E[Nearest edge<br/>fly-proxy]
  E -->|TLS handler, headers,<br/>pick Machine| W1[Worker in nearest<br/>healthy region<br/>fly-proxy]
  E -.->|WireGuard backhaul| W1
  W1 -->|local virtual interface| M1[Machine<br/>Firecracker microVM]
  E --> W2[Worker, other region]
  W2 --> M2[Machine]
  C[(Corrosion<br/>service catalog)] -.->|routing data| E
  C -.-> W1
  M1 -.->|state changes| F[flyd on worker]
  F -.->|gossip| C

Whiteboard diagram (rendered from the source mermaid)

In words (https://fly.io/docs/reference/fly-proxy/): the request lands on the closest edge server. Fly Proxy there checks the request, adds headers, redirects HTTP to HTTPS, and sends it through the WireGuard tunnel backhaul to the nearest healthy worker hosting your Machines. Fly Proxy on the worker hands it to a Machine over a local virtual interface, and the response retraces the path.

Layer 1: the edge (Anycast and Fly Proxy)

Fly.io says it will “announce global IP blocks from all of our datacenters over BGP”, and describes Anycast as a routing mechanism that connects clients to the nearest server advertising a block of IPs (https://fly.io/docs/networking/services/). A 2021 post (dated 2021-10-18) gives the recipe: “we have blocks of addresses, machines in a bunch of regions, BGP4 peering in each of them, and a Rust-based proxy that picks up connections for those addresses” (https://fly.io/blog/32-bit-real-estate/).

Fly.io’s own hardware is “roughly divided into two kinds of servers: edges, which receive incoming requests from the Internet, and workers, which run Fly Machines” (https://fly.io/blog/taming-rust-proxy/, 2025-02-26). A third type, the gateway, “terminates WireGuard connections that provide access to your private network, or 6PN” (https://fly.io/docs/security/shared-responsibility/).

Fly Proxy is “the omnipresent routing layer between the public Internet and the Fly.io platform”, and “Every Fly.io edge and worker runs Fly Proxy” (https://fly.io/docs/reference/fly-proxy/). Its jobs include:

Matching connections to apps. Shared IPv4 addresses are shared across apps and orgs, with “Routing is based on your app’s domain” (https://fly.io/docs/networking/services/).
Connection handlers. Most web services use http and tls handlers; Postgres gets pg_tls; handlers are optional and without them the proxy “just forwards TCP” (https://fly.io/docs/reference/fly-proxy/).
Headers your app sees. Fly-Client-IP, Fly-Region (where the connection was accepted), X-Forwarded-For and X-Forwarded-Proto (https://fly.io/docs/networking/request-headers/).
Autostop and autostart. The proxy stops or suspends at most one Machine per region per cycle, and only works on Machines that already exist (https://fly.io/docs/reference/fly-proxy-autostop-autostart/).
Dynamic routing. A fly-replay response header tells the proxy to replay the request to another region, Machine or app (https://fly.io/docs/reference/fly-proxy/).

How a Machine is chosen

The rule is: “Send traffic to the least loaded, closest Machine” and, on ties, choose randomly (https://fly.io/docs/reference/load-balancing/). Details:

Machine load	What happens
Above `hard_limit`	No new traffic is sent
At or above `soft_limit`, below `hard_limit`	Used only if all other Machines are also above `soft_limit`
Below `soft_limit`	Used when it is the closest Machine under `soft_limit`

Source: https://fly.io/docs/reference/load-balancing/. Closeness is the round-trip time between the receiving edge and the worker, measured constantly between all servers. “Cross-region routing only happens when all Machines in the local region are unhealthy or at their hard_limit” (https://fly.io/docs/reference/load-balancing/).

Fly Proxy is a Rust program, and it has had bad days

A 2025 post describes a bug where fly-proxy CPU spiked on edges in IAD because async Rust Futures polled in a loop. Bouncing the proxy cleared it temporarily (https://fly.io/blog/taming-rust-proxy/). The September 1, 2024 outage was worse. A new Machine with a “virtual service” option arrived, and “Within a few seconds every proxy in our fleet had locked up hard”, the “worst outage we’ve experienced” (https://fly.io/blog/corrosion/). Say: “Fly.io writes its failures up in public, and the fixes are specific, such as watchdogs on every Tokio program.” The watchdog claim is in https://fly.io/blog/corrosion/.

Layer 2: workers (Firecracker microVMs)

“Application code runs in Firecracker microVMs. These are lightweight, secure virtual machines based on strong hardware virtualization” (https://fly.io/docs/reference/architecture/). The same page says:

Dedicated-ish resources. “We make a best-effort attempt to dedicate hardware resources to only one microVM at a time”, so CPU cores should not see steal (https://fly.io/docs/reference/architecture/).
Big servers. Workers are “dedicated physical servers with 8-32 physical CPU cores and 32-256GB of RAM” (https://fly.io/docs/reference/architecture/).
Why not Docker. Fly.io chose Firecracker after surveying namespaces, gVisor and Kata, calling it “a VMM optimized for security” (https://fly.io/blog/sandboxing-and-workload-isolation/, 2020-07-29). A 2021 post says: “we transmogrify container images into Firecracker micro-VMs” (https://fly.io/blog/docker-without-docker/).
GPUs differ. GPU Machines use Intel’s Cloud Hypervisor instead, because it supports PCI passthrough (https://fly.io/blog/wrong-about-gpu/).

Image to VM

The 2021 post describes the pipeline: pull the OCI image into a server-local containerd on an LVM2 thin pool, check out a snapshot per VM, and boot it in Firecracker with a custom init “written in Rust” that mounts filesystems, reads an injected config, and starts the entrypoint (https://fly.io/blog/docker-without-docker/). This is four years old, so confirm it is still current before presenting it as today’s design [verify].

Orchestration: flyd, flaps and the market model

In 2023 Fly.io explained why it left Nomad: it “operates like a market”, with “Requests to schedule jobs are bids for resources” and workers as suppliers (https://fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/). The cast:

flyd. “the source of truth for all the VMs running on a particular worker”, with a local boltdb and state machines (https://fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/).
flaps. “a stateful proxy for all the flyd instances”; it is what the Machines API talks to. The public base URL is https://api.machines.dev (https://fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/, https://fly.io/docs/machines/api/working-with-machines-api/).
flyctl. The CLI, which takes over policy work such as multi-region deploys and spare Machines (https://fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/).

Placement consequences you can explain to a customer: the API “will contact the Machines API server (we call it flaps…), which will in turn reach out to all the worker servers we have in the region” and “try to make a smart choice” (https://fly.io/docs/machines/overview/). The docs warn that “Placement can fail!” (https://fly.io/docs/machines/overview/), and the scheduler post says jobs do not arrive and then sit on the book in a pending state (https://fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/). Per the 2023 post, a deploy creates spare Machines on other workers and the proxy starts one if a worker dies (https://fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/).

One detail has changed: the 2022 post says create requests used NATS broadcasts (https://fly.io/blog/fly-machines/), while a 2024 post says “our internal flyd API used to be driven by NATS; today, it’s HTTP” (https://fly.io/blog/jit-wireguard-peers/).

Pinned hardware and migration

Machines are pinned to hosts. Volumes make that sharper, because a volume is a slice of NVMe on the same server (https://fly.io/docs/volumes/overview/). For drains, Fly.io built “clone”: a new volume is created elsewhere and hydrated lazily over the network while the new Machine runs (https://fly.io/blog/machine-migrations/, 2024-07-30). The docs describe the customer-visible steps: lock by network ID, stop the Machine with its kill_signal, fork the volume, start a new Machine with the same Machine ID (https://fly.io/docs/reference/machine-migration/).

Layer 3: the WireGuard mesh and 6PN

“WireGuard is fundamental to how Fly.io works”, and “Practically everything that talks to anything else at Fly.io does so over WireGuard” (https://fly.io/blog/our-user-mode-wireguard-year/, 2022-02-09). The 2020 6PN post says the network is “fully connected through a WireGuard mesh” and uses small BPF programs to enforce that you cannot talk from one 6PN network to another (https://fly.io/blog/incoming-6pn-private-networks/).

For customers (https://fly.io/docs/networking/private-networking/):

Apps in one organization share a 6PN, a mesh of IPv6 WireGuard tunnels, “always available to apps by default” (https://fly.io/docs/networking/private-networking/).
Names under .internal resolve to started Machines only. Autostopped Machines drop out of DNS answers.
Flycast gives an app a private address that goes through Fly Proxy, so internal traffic gets load balancing and autostart (https://fly.io/docs/reference/fly-proxy/).
Laptops and outside services join through WireGuard gateways, which exist in only some regions (https://fly.io/docs/reference/regions/).

Layer 4: Corrosion, the state catalog

Fly Proxy “gets all its information about apps and Machines and services from corrosion, our service catalog that stores the state of pretty much everything on the Fly.io platform” (https://fly.io/docs/reference/fly-proxy/). The October 2025 post gives the design (https://fly.io/blog/corrosion/):

Servers own their state. “individual servers are the source of truth for their workloads”, unlike Kubernetes-style central databases (https://fly.io/blog/corrosion/).
Gossip, not consensus. “Corrosion is a Rust program that propagates a SQLite database with a gossip protocol”, built on SWIM, with QUIC between nodes and cr-sqlite for last-write-wins CRDT merging (https://fly.io/blog/corrosion/).
No leaders. It has “no locking, no central servers, and no distributed consensus” (https://fly.io/blog/corrosion/).
Previously Consul. Fly.io relied on HashiCorp Consul for routing for “longer than we should have” (https://fly.io/blog/corrosion/).

The post is also candid about failure modes. A nullable column added to a Corrosion table forced a backfill that “melts down the entire cluster”. An expired Consul certificate made every worker retry in a loop, and each retry wrote to Corrosion until “we’re literally saturating our uplinks” (https://fly.io/blog/corrosion/). Fixes include watchdogs, testing with Antithesis, checkpoint backups on object storage, and re-publishing the whole Machine record on every change.

Open item: the post says Fly.io is moving from one global cluster to “regionalization”, with a Corrosion cluster per region and a global cluster mapping apps to regions. Whether that is finished is not stated in the sources [verify].

How a deploy flows

fly deploy builds or fetches an image. The order is --image or [build] image, then the [build] method, then --dockerfile, then a local Dockerfile (https://fly.io/docs/launch/deploy/).
If the app has a public HTTP service and no IPs, deploy provisions a public IPv6 and a shared IPv4 Anycast address (https://fly.io/docs/launch/deploy/).
The image goes to Fly.io’s registry, e.g. registry.fly.io/my-app-name:deployment-... (https://fly.io/docs/machines/runtime-environment/).
Machines are created through flaps and flyd; in created state Fly.io reserves space, fetches the container and builds a root file system, then boots to started (https://fly.io/docs/machines/overview/).
The strategy decides rollout: rolling (default), immediate, canary or bluegreen (https://fly.io/docs/launch/deploy/).
Smoke checks watch each Machine for about 10 seconds after it starts, and a Machine that keeps restarting with a non-zero exit code stops the deploy (https://fly.io/docs/launch/deploy/).
State changes flow into Corrosion, and proxies learn the new routes (https://fly.io/docs/reference/fly-proxy/).

How Sprites differ

Sprites “are still Fly Machines” underneath, but with “No More Container Images”, pooled standby Sprites on every worker, and S3-compatible object storage as the root of disk (https://fly.io/blog/design-and-implementation/, 2026-01-14). The docs say data is written to NVMe, then backed up to durable object storage when the Sprite goes idle (https://fly.io/docs/sprites/). The sub-processor list names Tigris for “Object storage for Sprites” (https://fly.io/legal/sub-processors/).

Regions

Fly.io lists 17 regions, including ams, arn, cdg, dfw, ewr, fra, gru, iad, jnb, lax, lhr, nrt, ord, sin, sjc, syd and yyz. Volumes and Machines are tied to the region where they were created (https://fly.io/docs/reference/regions/).

Questions to ask a customer

Where are your users, and how many Machines run per region? (Load and closeness decide routing: https://fly.io/docs/reference/load-balancing/.)
Does anything depend on one volume? (No built-in replication: https://fly.io/docs/volumes/overview/.)
Do you scale by count or by autostop? (Autostop never creates Machines: https://fly.io/docs/reference/fly-proxy-autostop-autostart/.)

← What Fly.io is · Apps, Machines and volumes →