The network edge: Fly Proxy, IPs, private networking and domains

Public traffic lands on the nearest edge, where Fly Proxy terminates TLS and carries the request to a healthy Machine over WireGuard. Your apps also share a private IPv6 network (6PN), and Flycast puts Fly Proxy in front of private apps.

This page is drawn from https://fly.io/docs/reference/fly-proxy/, https://fly.io/docs/networking/services/, https://fly.io/docs/networking/private-networking/, https://fly.io/docs/networking/flycast/, https://fly.io/docs/networking/custom-domain/, https://fly.io/docs/networking/dynamic-request-routing/, https://fly.io/docs/networking/request-headers/, https://fly.io/docs/networking/egress-ips/ and https://fly.io/docs/about/pricing/.

Fly Proxy and Anycast

“Fly Proxy is the omnipresent routing layer between the public Internet and the Fly.io platform and within the global WireGuard mesh that encompasses everything that runs on Fly.io.” (https://fly.io/docs/reference/fly-proxy/) Every edge and worker runs it (https://fly.io/docs/reference/fly-proxy/). It is a Rust-based proxy named fly-proxy (https://fly.io/docs/reference/architecture/).

Fly announces global IP blocks from its datacenters over BGP, known as Anycast, so clients reach the “nearest” server advertising the block (https://fly.io/docs/networking/services/).

flowchart LR
  U[User] --> E[Nearest edge<br/>Fly Proxy]
  E -->|WireGuard backhaul| W[Worker<br/>Fly Proxy]
  W --> M[Machine]

What happens to a public request (https://fly.io/docs/reference/fly-proxy/):

1. The request lands on the closest edge server.
2. Fly Proxy adds headers and redirects HTTP to HTTPS.
3. It routes through the WireGuard tunnel to the nearest healthy worker hosting your Machines.
4. Fly Proxy on the worker hands it to a Machine over a local virtual interface.

Example from the architecture page: with users in Dallas and an available microVM in Chicago, traffic is accepted in Dallas, TLS is terminated, and the connection reaches the microVM over WireGuard (https://fly.io/docs/reference/architecture/).

Load balancing

Fly Proxy sends traffic to “the least loaded, closest Machine”, picking randomly on ties (https://fly.io/docs/reference/load-balancing/). Load comes from soft_limit and hard_limit in your concurrency settings (https://fly.io/docs/reference/load-balancing/).

Closeness is the measured RTT between the receiving edge and the worker, and cross-region routing happens only when local Machines are unhealthy or at hard_limit (https://fly.io/docs/reference/load-balancing/).

Public IPs

(https://fly.io/docs/networking/services/ and https://fly.io/docs/about/pricing/)

Apps created with fly machine commands or the Machines API need fly ips allocate-v6 and fly ips allocate-v4 --shared run by hand (https://fly.io/docs/networking/services/).

Reasons the docs give for a dedicated IPv4 (https://fly.io/docs/networking/services/):

• Your app uses a non-HTTP protocol that does not use TLS.
• Your app needs UDP.
• You want raw TCP and handle TLS yourself.
• You expose Fly Postgres over TLS externally.

Shared IPv4 is recommended unless you have an explicit need (https://fly.io/docs/networking/services/). Say: “Start on shared IPv4 and IPv6. Buy a dedicated address only for UDP, raw TCP or an allowlist you cannot avoid.” The docs give a no-downtime path from dedicated back to shared: allocate shared, confirm with curl --resolve, wait for DNS caches, then release the dedicated address (https://fly.io/docs/networking/services/).

Ports, handlers and what is exposed

Nothing is exposed unless a service exposes it. The proxy listens on all ports and Anycast addresses, so ports look open, but “You’re locked down by default.” (https://fly.io/docs/networking/services/)

Connection handlers go in [[services.ports]] (https://fly.io/docs/networking/services/):

With no handlers, TCP passes through as-is, which is how you terminate TLS yourself (https://fly.io/docs/networking/services/). force_https returns a 301 and can only be set on HTTP handlers (https://fly.io/docs/networking/services/). For HTTP/2 to the app (gRPC), enable h2_backend; the app must speak h2c (https://fly.io/docs/networking/services/).

Bind public services to 0.0.0.0:<port>, except UDP, which binds to fly-global-services (https://fly.io/docs/networking/app-services/). The UDP rules: a dedicated IPv4 is required, the port is not rewritten, and “We don’t support UDP over public IPv6” (https://fly.io/docs/networking/udp-and-tcp/). WebSockets work with an [http_service] and idle connections stay open, but deploys, proxy restarts and autostop close them with close code 1012 on the client, so clients must reconnect (https://fly.io/docs/networking/websockets/).

A warning to repeat: if a service is configured and the app has a public IP, “that service is exposed to the whole internet” (https://fly.io/docs/networking/app-services/).

Request headers

The HTTP handler adds these (https://fly.io/docs/networking/request-headers/ and https://fly.io/docs/networking/services/):

Request IDs: every Fly.io response carries a fly-request-id header, and it is “the most useful thing you can send” for an API problem (https://fly.io/docs/reference/feedback-api/). The Machines API can suppress Fly headers with pristine (https://fly.io/docs/machines/api/machines-resource/). Ask customers for fly-request-id on every bug report.

Clients can steer routing: Fly-Prefer-Region, Fly-Prefer-Instance-Id, Fly-Force-Instance-Id (https://fly.io/docs/networking/services/), and fly-force-region, which, unlike fly-prefer-region, will not fall back to the nearest region (https://fly.io/docs/networking/dynamic-request-routing/).

fly-replay

When your app answers with a fly-replay header, Fly Proxy redelivers the original request to another region, Machine or app in your organization (https://fly.io/docs/networking/dynamic-request-routing/). Fields include region, instance, prefer_instance, app, timeout and fallback (https://fly.io/docs/networking/dynamic-request-routing/).

fly-replay: region=sjc
fly-replay: app=my-worker;timeout=10s;fallback=force_self

(https://fly.io/docs/networking/dynamic-request-routing/)

Use cases listed: sending writes to the primary database region, load balancing between Machines, cross-app routing and sticky sessions (https://fly.io/docs/networking/dynamic-request-routing/). Limit: “Requests larger than 1MB cannot be replayed” (https://fly.io/docs/networking/dynamic-request-routing/). An app that is only a fly-replay target does not need its own IP (https://fly.io/docs/networking/app-services/).

6PN, .internal DNS and Flycast

Apps in an organization are linked by a mesh of WireGuard tunnels over IPv6 called a 6PN, available by default (https://fly.io/docs/networking/private-networking/). Other organizations cannot reach it unless you explicitly allow it, for example with Flycast (https://fly.io/docs/networking/private-networking/).

.internal DNS (https://fly.io/docs/networking/private-networking/):

Stopped Machines, including those autostopped by Fly Proxy, are not in AAAA answers (https://fly.io/docs/networking/private-networking/). The DNS server is fdaa::3 (https://fly.io/docs/networking/private-networking/). Bind private services to fly-local-6pn (https://fly.io/docs/networking/private-networking/). 6PN addresses are not static and change on migration or org move; use the .vm. name (https://fly.io/docs/networking/private-networking/). Custom 6PNs isolate tenants (https://fly.io/docs/networking/custom-private-networks/).

Flycast routes private traffic through Fly Proxy, so private apps get geographic load balancing, and Fly Proxy can autostop and autostart Machines based on requests (https://fly.io/docs/networking/flycast/). Start with fly launch --flycast and reach it at my-app-name.flycast (https://fly.io/docs/networking/flycast/). Requirements: bind 0.0.0.0, define a service, and skip force_https because “Flycast is HTTP-only.” (https://fly.io/docs/networking/flycast/) Remove public IPs from private apps (https://fly.io/docs/networking/app-services/).

TLS and custom domains

Fly Proxy accepts only TLS 1.2 and 1.3 (https://fly.io/docs/networking/tls/). It terminates TLS by default using Let’s Encrypt certificates or your own (https://fly.io/docs/security/tls-termination/).

Add a domain with fly certs add example.com, then set DNS (https://fly.io/docs/networking/custom-domain/). A and AAAA records are recommended, CNAME suits subdomains, and an apex CNAME needs flattening (https://fly.io/docs/networking/custom-domain/). Validation uses TLS-ALPN (preferred), HTTP-01, or DNS-01 for wildcards (https://fly.io/docs/networking/custom-domain/). A _fly-ownership TXT record covers CDN and proxy cases (https://fly.io/docs/networking/custom-domain/). Imported certificates use fly certs import (https://fly.io/docs/networking/custom-domain/).

Pricing: $0.10/month per single hostname and $1/month per wildcard, first 10 single-hostname certificates free (https://fly.io/docs/about/pricing/).

Cloudflare: grey cloud (DNS only) is simplest; with the orange cloud proxy, add _fly-ownership, and wildcard certificates cannot be issued automatically through the proxy, so import an Origin Certificate (https://fly.io/docs/networking/understanding-cloudflare/).

Access and egress

WireGuard. fly wireguard create produces a tunnel config for a laptop or outside service, with the private key embedded and not recoverable (https://fly.io/docs/networking/private-networking/). Pick a gateway region from fly platform regions (https://fly.io/docs/networking/private-networking/).

Egress IPs. “A Machine’s outbound IP is liable to change without notice.” (https://fly.io/docs/networking/services/) The docs “discourage the use of our outbound IPs to bypass firewalls” (https://fly.io/docs/networking/services/). For allowlists, allocate app-scoped static egress IPs with fly ips allocate-egress -r <region>; one pair is needed per region, at $3.60/month per IPv4, with IPv6 included (https://fly.io/docs/networking/egress-ips/). Limits: 64 Machines per IP and 1024 concurrent connections per destination IP per Machine (https://fly.io/docs/networking/egress-ips/).

Data transfer

Inbound data is free; outbound to the internet is $0.02/GB in North America and Europe, and cross-region private traffic is $0.006/GB there (https://fly.io/docs/about/pricing/).