THE TRENCHES
10 MIN TALK

▲ Platform Engineering :: The Field Talk

The Edge, Not the Region

AWS asks you to pick a region and build inside it. Cloudflare runs every service on every machine in 330+ cities, so there is nothing to pick. Ten minutes, first principles: how the network works, 20 services scored fairly against AWS, and the Terraform provider that turns all of it into a reviewable plan.

Boyan Balev The Trenches Field Talk Series

00 / The Frame 0:00Two physics, one scoreboard

We run both clouds and will keep running both. The real question, asked per workload: does it need distribution (low latency everywhere, cheap delivery) or gravity (large state, heavy compute, private networks)? Every instrument below scores against that question.

330+cities, one deploy target
~38AWS regions you choose from
1IP announced from all of them
$0egress from R2

01 / The Network 0:30Anycast: one IP, announced everywhere

First principle: an IP address is not a location. It is a destination that routers agree on through BGP. If 330 sites announce the same IP, each router on the internet picks its own shortest path to it, so every packet lands at the nearest site. Routing does the load balancing. Nobody operates it.

How a packet finds the edge play both modes

Act 1: every PoP announces the same IP into BGP. Act 2 happens inside the routers: each picks its shortest path, with no coordinator. Act 3: a request from Sofia follows that path to the nearest node. Switch modes to see the same request against one address in one region.

Anycast mode. Press Announce to watch every node claim the same address.

Why this matters: every machine answers the same IP and runs the same software. Any new product is instantly global. Any node can serve any request. There is no routing tier to build, scale, or break. Everything else in this talk follows from this design.

02 / Anatomy 1:30One request, one machine, six gates

Second principle: reject cheaply, spend late. The six gates on an edge machine are ordered by the cost of saying no. Packets are dropped before TCP state exists, requests are blocked before compute runs, and cache answers before the origin is asked. The ordering is the security model and the cost model at the same time.

The request pipeline step through the gates

All six gates run on the one machine that anycast already picked. Each step names the products at that gate and why the gate sits where it sits.

A packet arrives at the nearest PoP. Press next.

The origin is the last gate because it is the only one that costs a cross-continent round trip. Every earlier gate exists to avoid that trip.

03 / The Catalog 2:30Twenty services, twenty counterparts

For the serverless part of a platform, Cloudflare now has a counterpart to almost everything in the region. Click a row for the verdict. The gist column states the disagreement, not the description.

The service ledger filter · click a row

CloudflareAWS counterpartThe disagreement
Default read: the edge wins where distribution and egress dominate; the region wins where compute depth, data size, and private networking dominate. Click any row.

04 / Isolates 4:00Workers vs Lambda: what a cold start physically is

First principle: a cold start is the time it takes to manufacture isolation. Lambda isolates with hardware virtualization: each concurrent execution gets a Firecracker microVM with its own kernel and runtime, which must boot when cold, typically 100 ms to over 1 s. Workers isolate with V8 sandboxes: thousands share one process that never stops, and a new isolate is created in under 5 ms. Same goal, different manufacturing cost.

Two isolation factories drag the traffic up

Left: one edge machine, one process, isolates created inside it. Right: the region creating one microVM per concurrent execution. Same traffic, different physics.

1 5,000
100 concurrent

The isolate pays for speed with hard limits: 128 MB of memory, JavaScript or WASM only, no local disk, CPU-time budgets. The microVM pays for freedom with startup cost: up to 10 GB of memory, any runtime, 15 minutes of wall clock, VPC access. The rule: request-shaped logic (auth, routing, rendering, API glue) fits isolates. Machine-shaped work (batch jobs, large memory, filesystems) fits microVMs. Problems start only when one shape is shipped into the other's runtime.

05 / Data 5:00Storage is a placement decision

First principle: for distributed state, the design decision is where the authoritative copy lives. Consistency, latency, and price all follow from that placement. Four primitives, four placements:

Where does the truth live? pick a primitive

Dots are PoPs. The solid marker is the authoritative copy; hollow markers are copies. The placement explains the guarantee.

The egress bill, computed live two sliders, one invoice

S3: $0.023 per GB stored, $0.09 per GB sent to the internet. R2: $0.015 per GB stored, $0 sent. Cloudflare can price egress at zero because delivery rides the network it already operates. It is a different business model, not a discount.

stored20 TB
egress60 TB

06 / DNS 6:15The orange cloud: one boolean, two topologies

First principle: DNS decides what address the world learns for your hostname. Answer with the origin's IP (grey cloud) and Cloudflare is only a fast phone book. Answer with anycast edge IPs (proxied = true) and every request enters the six-gate pipeline from section 02, while the origin firewall can allow only Cloudflare's ranges.

proxied = true flip the cloud

# provider v5 :: the boolean that changes the topology
resource "cloudflare_dns_record" "app" {
  zone_id = var.zone_id
  name    = "app"
  type    = "CNAME"
  content = "origin-lb.eu-west-1.elb.amazonaws.com"  # the origin can be AWS
  proxied = true                                     # route through the edge
  ttl     = 1                                        # auto: the edge owns the TTL
}

# attach a domain to a worker in code: no records created behind your back
resource "cloudflare_workers_custom_domain" "api" {
  account_id  = var.account_id
  zone_id     = var.zone_id
  hostname    = "api.example.com"
  service     = cloudflare_workers_script.edge_api.script_name
  environment = "production"
}

Three facts to carry. Registrar: Cloudflare sells domains at wholesale cost with no markup, so purchase, DNS hosting, and proxying live in one account and one provider block. Partial (CNAME) setup: if you cannot move nameservers, you can CNAME individual hostnames to Cloudflare and still get the proxy; Enterprise only, fewer features. UI vs code: attaching a Worker or Pages domain in the dashboard creates DNS records and certificates implicitly; the resource above does the same thing explicitly, in review.

07 / CDN 7:00Cache is a hierarchy, and rules are code

First principle: a cache converts a global round trip into a local read. Tiered Cache adds a second idea: put one upper tier between the 330 PoPs and the origin, so each miss reaches the origin once instead of 330 times.

Miss, hit, then a new city asks fire the three requests

Same object, three requests. Watch which hops each one needs and which it skips.

Nothing cached yet. Fire request 1.

Why tiered: without it, every city misses independently and the origin absorbs up to 330 first-requests per object. With it, the upper tier takes the miss once and every other PoP fetches from inside the network. Cache Reserve extends the same idea into R2 for rarely requested objects.
# cache strategy as a reviewed diff :: the v5 rules engine
resource "cloudflare_ruleset" "cache" {
  zone_id = var.zone_id
  name    = "cache-rules"
  kind    = "zone"
  phase   = "http_request_cache_settings"

  rules = [{
    expression = "http.request.uri.path.extension eq \"html\""
    action     = "set_cache_settings"
    action_parameters = {
      cache    = true
      edge_ttl = { mode = "override_origin", default = var.edge_ttl_override }
    }
    enabled = true
  }]
}

08 / The Shield 7:45DDoS: divide the attack by 330

First principle: anycast spreads attackers the same way it spreads users. Each bot's traffic enters at the PoP nearest to that bot, so a terabit attack arrives already divided into hundreds of slices. This is why mitigation is unmetered and included on every plan: the network's size is the mitigation. AWS sells the equivalent posture as Shield Advanced at about $3,000 per month.

The attack, divided scale it up

One slider: total attack size. Compare the load on a single origin (40 Gbps link assumed) with the load on each of 330 PoPs. What survives L3/4 is handled by the WAF at gate 3.

10 Gbps6 Tbps
400 Gbps

09 / The Provider 8:30v5: the API wearing HCL

The v5 provider is generated from Cloudflare's OpenAPI schemas. Three consequences. Resources map one to one to API objects. Configuration is attributes (= syntax) rather than nested blocks. Names follow the API: cloudflare_record became cloudflare_dns_record, and zone settings are one cloudflare_zone_setting per setting. New API features reach the provider almost immediately. The cost is occasional sharp edges from generated schemas, which is why the module layer exists: wrap the raw resources once, and product teams only touch the wrapper.

Fifteen lines in, an estate out the composition

A product team writes the module call on the left. The service module instantiates the shared modules. Each module owns raw v5 resources. Each resource is one API object. The plan you review is this graph.

module "service" hostname, origin, ttl, waf = true zone_settings dns + cache-rules waf-zone load-balancer raw v5 cloudflare_dns_record cloudflare_ruleset cloudflare_zone_setting cloudflare_load_balancer API 1 : 1 accounts / tokens / modules / services / teams :: the whole estate is this shape
State closes the loop: remote state in a locked backend, plan in CI on every pull request, apply from one pipeline identity. The dashboard is read-only for humans. A clicked change is drift; the next plan reverts it or fails. To adopt resources created before the code existed: cf-terraforming generates the HCL, then terraform import cloudflare_dns_record.www $ZONE_ID/$RECORD_ID brings each into state.

10 / Capstone 9:15terraform apply, visualized

Everything above, applied once. Terraform derives the execution order from the references between resources, so the architecture diagram and the plan are the same object. They cannot drift apart.

One service, six resources, plan order step the apply

plan: 6 to add
ZONE settings, tiered cache DNS proxied = true s06: the front door CACHE RULESET edge_ttl override s07: rules as code WAF managed + custom s08: the shield WORKER + DOMAIN isolates on the route s04: request-shaped LOGPUSH edge logs → R2 s05: zero-egress sink

Plan ready. Six resources, dependency-ordered. Press next.

The origin behind all of this is often an AWS load balancer. The edge and the region are layers, not rivals. The only real argument is which layer owns which job.

11 / The Meta Lesson 9:45The verdict, and the lookup table

Distribution problems: the edge. Global latency, egress-heavy delivery, request-shaped compute, security included. Gravity problems: the region. Large state, long compute, private networking, residency. Both: Terraform. One provider each, one repository, one reviewable plan instead of two dashboards.

users everywhere, origin in one place→anycast: routing is the balancer cold starts poison the p99→isolates, not microVMs heavy CPU, big memory, long jobs→Lambda / the region, honestly the egress line owns the invoice→R2: $0 out coordination without a lock service→Durable Objects: one truth per key origin IP exposed to the internet→proxied = true, firewall the rest every PoP misses independently→tiered cache: miss once terabit attack, one address→divide by 330, included clickops drift, invisible changes→UI observes, code decides raw v5 resources are sharp→wrap once, consume the wrapper

Name the property the workload needs, distribution or gravity, and the vendor picks itself. The logos rotate. The physics do not.

See you in the trenches.