Home Resume
Home Blog AWS Architecture Series #13 — CloudFront, Origin Shield and WAF: When the CDN Is Just an Expensive Proxy…
AWS AWS Architecture Series

AWS Architecture Series #13 — CloudFront, Origin Shield and WAF: When the CDN Is Just an Expensive Proxy

The cache key decides whether a CDN caches anything at all. Get it wrong and every edge layer becomes a pass-through — and Origin Shield bills you for the privilege.

Business Challenge

An e-commerce platform put CloudFront in front of its product pages and API eighteen months ago. The origin is an ECS service behind an ALB in eu-west-1. The expectation was the usual one: the CDN absorbs the reads, the origin handles the writes, and the fleet stops scaling with traffic.

Origin request volume never dropped. During the last sale the origin scaled to the same instance count it would have needed with no CDN at all, and the CloudFront bill arrived on top of it. The team had bought a cache and were paying for a proxy.

1The cache key had been built to be safe, not to cache

When the distribution was created, someone forwarded User-Agent so the origin could do device detection, and the session cookie so personalisation would work. Both landed in the cache key. User-Agent alone has thousands of variations; a session cookie is unique per user. Between them, effectively every request produced a distinct cache key. The cache was full of objects with one hit each.

Fix

Only values that change the response the origin returns belong in the cache key.

2The origin genuinely needed those values — but not in the key

The forwarding was not wrong, only the mechanism. CloudFront has a second policy type for exactly this: an origin request policy sends headers, cookies and query strings to the origin without including them in the cache key. Analytics and telemetry values reach the origin, and the cache stays whole.

Fix

Cache policy decides what varies the cached object. Origin request policy decides what the origin gets told.

3Origin Shield was switched on to fix a caching problem

Faced with origin load, the team enabled Origin Shield — a reasonable-sounding move, since it collapses duplicate origin requests. But request collapsing only helps when many requests want the same object, and a cache key that is unique per user produces almost no duplicates to collapse. AWS says so plainly: Origin Shield is not a good fit for low-cacheability or infrequently requested content.

Fix

Fix the cache key first. Origin Shield multiplies a working cache; it cannot create one.

4And the TTL made every request billable

Product pages were set to a 300-second TTL to keep pricing fresh. For Origin Shield billing, a GET or HEAD with a TTL below 3,600 seconds is treated as a dynamic request, and dynamic requests always count Origin Shield as an incremental layer. The entire read path was billed at the dynamic rate while delivering the collapsing benefit of almost none of it.

Fix

Know that the 3,600-second boundary exists before choosing a TTL, not after reading the invoice.

Architecture

A CloudFront request passes through several layers, and it is tempting to reason about them independently. They are not independent: one decision governs all of them.

Diagram: a viewer request passing through AWS WAF, the edge location cache, a regional edge cache, Origin Shield and finally the origin. Side panels explain that the cache key defaults to distribution domain plus URL path, that an origin request policy forwards values without adding them to the cache key, and that Origin Shield bills GET requests with a TTL under 3,600 seconds as dynamic.
The layers are sequential, but the cache key decides how many of them can do anything. Get it wrong and each one becomes a toll booth on the way to the origin.

The default cache key is smaller than people expect

Out of the box, the cache key is the distribution domain name and the URL path. That is all. Query strings, headers and cookies are excluded until a cache policy adds them. Two requests for the same path with different query strings, different User-Agent values and different session cookies are, by default, a cache hit.

This matters because the default is the good configuration for most static content, and almost every cache-hit problem is something a human added on purpose.

Two policies, two different questions

Cache policy

"Does this value change the response?" If yes, it belongs in the cache key — a different value should produce a different cached object.

Origin request policy

"Does the origin need to see this?" If yes but it does not change the response, forward it without touching the cache key.

Conflating the two is the single most common CloudFront mistake, and it is easy to make because the safe-looking option — forward everything — is the one that destroys the cache.

Origin Shield collapses; it does not cache into existence

With Origin Shield enabled, every origin request from every CloudFront layer funnels through a single additional cache in one Region. Requests for an object that is not there are consolidated, so a burst of identical misses can become as few as one origin request. That is genuinely powerful for a live event, just-in-time packaging, or a bandwidth-constrained on-premises origin.

1

Choose the Region by proximity to the origin

If the origin sits in one of the thirteen Regions where Origin Shield is offered, use that same Region. Otherwise pick the nearest from AWS's mapping table.

2

Same-Region requests skip it

Requests that reach the regional edge cache in the same Region as Origin Shield bypass the extra layer, and are not billed for it.

3

Check the logs for what actually happened

OriginShieldHit appears in the x-edge-detailed-result-type field. A request served by the regional edge cache acting as Origin Shield logs as a plain Hit, so the two are not interchangeable when measuring.

4

Know what moves with it

Lambda@Edge origin-facing triggers run in the Origin Shield Region. gRPC is not supported and bypasses the layer entirely.

WAF sits in front of all of it

A web ACL associated with the distribution inspects requests before they are served from cache or forwarded to the origin, so blocked traffic costs neither a cache lookup nor an origin request. Worth knowing if you are on one of the newer flat-rate pricing plans: those bundle CloudFront, WAF, DDoS protection, bot management, Route 53, log ingestion, TLS and edge compute into a monthly price — and while on such a plan, a web ACL must remain associated with the distribution. You can change the rules, but you cannot detach it without moving back to pay-as-you-go.

Why This Architecture Holds Up

Cache hit ratio is a design output, not a metric to chase

Teams treat hit ratio as something to be improved by tuning. It is mostly determined the moment the cache policy is written. If the key contains a value with high cardinality, no amount of TTL adjustment or extra caching layers will help, because the layers never see the same key twice.

AWS's own example is worth internalising: Accept-Language arrives as en-US,en, en,en-US, en-US, en and en-US. Four spellings of "English", four cached copies of one object. The recommendation is not to normalise the header but to put the language in the URL/en-US/content/… — so the path carries the variation and the cache key stays clean.

The test for any cache key candidate

How many distinct values will this take across real traffic? Two or three is fine. Thousands means you are not caching, you are indexing.

The ordering matters more than the components

Every layer here is worth having, but only in sequence. WAF first, because rejecting traffic early costs nothing downstream. Then a cache key narrow enough that objects are actually reused. Then, and only then, Origin Shield to collapse whatever misses remain.

Enabled in the wrong order, Origin Shield becomes a metered pass-through: an extra network hop and a per-request charge applied to traffic that was never going to hit anyway.

The 3,600-second line

Below an hour, GET and HEAD are billed as dynamic through Origin Shield. Plenty of sensible TTLs sit just under it. If you need short TTLs for freshness, that is a legitimate choice — just make it knowing Origin Shield's economics change on the other side of that boundary.

Where Origin Shield genuinely earns its cost

The cases AWS names are specific and they share a shape: many viewers wanting the same object at the same time, from an origin that finds serving it expensive. Live video with just-in-time packaging. On-the-fly image processing. An on-premises origin with a fixed bandwidth ceiling. Multiple CDNs in front of one origin, where Origin Shield also gives you a common cache key across them.

Notice that none of those is "our origin is under load". Origin load is a symptom; these are diagnoses.

Key Architecture Decisions

Decision Choice Reasoning
What goes in the cache key Only values that change the response Every added value multiplies the objects cached. High-cardinality values turn the cache into a per-user store with one hit each.
Values the origin needs anyway Origin request policy Forwards headers, cookies and query strings to the origin without fragmenting the cache. This is the fix for "but we need User-Agent".
Language and locale variation Put it in the URL path Header spellings vary; paths do not. Moves the variation somewhere the cache key handles cleanly.
Origin Shield Only once content is cacheable It collapses duplicate requests. With a per-user cache key there are no duplicates to collapse, and the charge lands anyway.
Origin Shield Region Nearest to the origin, from the supported thirteen Same-Region requests bypass and are unbilled, and Lambda@Edge origin-facing triggers follow the Region you pick.
WAF placement Web ACL on the distribution Blocked traffic never reaches cache or origin. On flat-rate plans the association is mandatory and cannot be removed.

What to measure before changing anything

Two numbers settle most arguments here. The first is cache hit ratio per behaviour, not per distribution — an average across a static path and an API path tells you nothing. The second is the number of distinct cache keys being generated, which you can approximate from access logs; if it approaches your request count, the cache key is the problem and nothing downstream will fix it.

The cheapest experiment

Create a second cache behaviour for one static path with a minimal cache policy and compare its hit ratio to the existing one. It takes minutes, changes nothing for users, and usually ends the debate about whether the cache key is at fault.

Closing Thought

A CDN is often described as something you put in front of an application. It is more accurate to say a CDN is a bet that many requests are the same request — and the cache key is where you declare what "the same" means. Everything else at the edge is machinery serving that one definition.

Which is why the failure in this scenario was not a misconfiguration so much as a category error. Forwarding User-Agent and a session cookie was a sensible instinct about what the origin needed. It just got expressed in the one place where it silently told CloudFront that no two requests are ever alike. Everything downstream then behaved perfectly correctly, at considerable expense.

The honest caveat: some content genuinely cannot be cached, and no cache key rewrite changes that. Authenticated API responses, personalised pages, anything with per-user pricing. For those paths the right answer may be no caching layer at all — a separate cache behaviour with caching disabled, routed straight to the origin, so the uncacheable traffic stops distorting the numbers for the traffic that is.

Next in this series

Storage — S3 storage classes and lifecycle economics at scale. Where Intelligent-Tiering earns its monitoring fee and where it quietly does not, what retrieval actually costs when you need the data back in a hurry, and the minimum-duration charges that make "just move it to Glacier" more expensive than leaving it alone.

Official AWS Reference

  • Understand the cache key — the default cache key, the effect of adding high-cardinality values, the Accept-Language duplication example, and the guidance to use an origin request policy for values the origin needs but the cache should ignore.

Comments

How was your experience?
Your feedback helps improve this site.
PoorExcellent