What it costs to have kept everything

Business Challenge

The last post ended with an instruction: stop using the default flow log format, add the packet-level addresses and the egress path, and consider a one-minute aggregation interval. Every one of those choices increases volume, and AWS says so plainly about the last of them — a one-minute maximum aggregation interval “produces a higher volume of flow log records” than ten.

That advice is only responsible if it comes with the bill attached. So this post is the bill.

A security team reviews its logging spend after a year of doing this properly. Flow logs with a custom format on every VPC, CloudTrail organisation trail, Resolver query logs, application logs from a few hundred functions and services. The number is larger than expected and the instinct is immediate and wrong: cut retention. Twelve months becomes three, somebody signs it off as a quick win, and the saving is a fraction of what was forecast.

It was always going to be. Retention is the cheap half.

1Ingestion is paid once and dominates early

CloudWatch Logs charges $0.50 per GB to ingest into the Standard class and $0.03 per GB per month to store. A gigabyte ingested and kept for a year therefore costs $0.86, and more than half of that was spent on the day it arrived.

Cutting retention from twelve months to three saves nine months of storage on that gigabyte — twenty-seven cents against a total of eighty-six. The ingestion is already spent and no retention policy recovers it. That is why the quick win disappoints: it is applied to the smaller number.

Fix

Model the two separately before choosing a lever. If the archive is young, the lever is what you ingest, not how long you keep it.

2S3 does not avoid the delivery charge

“We send flow logs to S3 because CloudWatch Logs is expensive” is the most common sentence in this area and it is half wrong. Flow logs, WAF logs and the rest are vended logs, and the vended-logs delivery rate applies across destinations — CloudWatch Logs, Amazon S3 and Amazon Data Firehose alike — with additional charges applying at the destination on top.

So the delivery is billed either way. What S3 changes is everything after delivery: storage at a lower rate, lifecycle transitions, and query through Athena rather than Logs Insights. Those are real savings. They are simply not the saving the sentence claims.

Fix

Choose the destination on what you will do with the data, not on an avoided charge that is not avoided.

3The tiers reward consolidation by a factor of ten

Vended-logs delivery is tiered: $0.50 per GB up to 10 TB, $0.25 from 10 to 30 TB, $0.10 from 30 to 50 TB, and $0.05 above 50 TB. The top tier is a tenth of the first, and the tiers reset at the start of every month.

An estate that delivers logs per-account, into per-account destinations, keeps every account in the most expensive tier forever. The same total volume consolidated into one delivery path climbs the tiers and pays a fraction for the marginal terabyte. This is the cost argument for a central log archive, and it is usually stronger than the governance argument that gets made for it.

Fix

Consolidate delivery, then measure the effective blended rate rather than the headline one.

Architecture

There are three prices in a log archive and they behave differently. Confusing them is what makes logging budgets hard to reason about.

The three prices

Delivery and ingestion · once per GB

$0.50/GB into the CloudWatch Logs Standard class, $0.25/GB into Infrequent Access. Vended logs are billed on their own tiered scale, from $0.50/GB to $0.05/GB, whichever destination they go to.

Retention · every month

$0.03/GB/month in CloudWatch Logs. Small per unit, and the only one that compounds. It is what an archive costs in year five, when the ingestion of year one is long since paid.

Query · per use

$0.01/GB scanned by Logs Insights. Usually negligible, occasionally not: an unbounded query across a year of flow logs scans the whole year.

CloudTrail, separately

Management events are free for the first copy. Additional copies are $2.00 per 100,000 events, and data events are $0.10 per 100,000. The second trail is where CloudTrail bills usually come from.

The crossover, and why it matters more than the rate

One gigabyte, Standard class, kept twelve months: fifty cents of ingestion and thirty-six cents of storage. Ingestion is the majority. Keep the same gigabyte five years and storage passes it. Every archive therefore has a point where the dominant cost changes hands, and where it sits depends entirely on your retention policy rather than on anything AWS charges.

The practical consequence is that the two levers are not interchangeable and they act on different time horizons. Reducing what you ingest lowers both numbers, but only for data that has not arrived yet. Reducing retention lowers only the smaller number, and does so for data you have already paid to ingest — which is the closest thing to a sunk cost in this whole model.

CloudTrail's free tier is per copy, not per account

Management events are logged by default at no charge for the first copy. A second delivery of the same events — an organisation trail alongside a pre-existing account trail, or a trail added for a specific tool — is billed at $2.00 per 100,000 events. That is a rate high enough to notice and structured in a way that makes duplication easy to create by accident.

Why This Architecture Holds Up

The cheap ingestion class is a one-way door

Infrequent Access halves ingestion, from $0.50 to $0.25 per GB, which on a large estate is the single largest available saving. It comes with the constraints from #12, and they are worth restating because a cost review is exactly when somebody proposes this without them: the class supports no subscription filters, no metric filters and no Live Tail, and a log group's class cannot be changed after it is created.

Two further details decide whether the trade is good. The saving is on ingestion only — AWS states the classes differ in ingestion cost alone, with identical storage and Logs Insights charges — so on an old archive the discount applies to the part that has already stopped mattering. And because the class is immutable, the decision is made per log group at creation, forever, on a guess about how that log group will be used.

The good version of this trade

Infrequent Access fits log groups that are written constantly and read rarely, by a human, after an event — which is a fair description of most flow log and audit data. It fits badly anywhere a metric filter drives an alarm, because that capability is simply absent. Sorting log groups by that question is a better first pass than sorting them by size.

The tiers make the archive account a cost decision

Series #41 argued for a dedicated log archive account on governance grounds: one place, one retention policy, one set of permissions. The vended-logs tiering adds an independent argument that happens to point the same way. Delivery billed at $0.50 for the first ten terabytes and $0.05 above fifty means the blended rate falls sharply with consolidation, and the tiers reset monthly, so this is a recurring benefit rather than a one-off.

It is unusual for the governance answer and the cost answer to agree this cleanly. When they do, the argument is worth making on both grounds, because the audience for each is different and the one that lands is not always the one you expect.

Key Architecture Decisions

Decision	Choice	Reasoning
First lever on a young archive	What you ingest	Ingestion is the majority of a gigabyte's first-year cost. It is also the only lever that lowers both numbers at once.
First lever on an old archive	Retention, and storage class at the destination	Once ingestion is years behind you, the recurring charge is what remains. Cutting it earlier would have saved less than it does now.
Destination	Chosen on use, not on avoiding delivery charges	The vended-logs rate is the same to CloudWatch Logs, S3 and Firehose. Only the post-delivery costs differ.
Delivery topology	Consolidated into the log archive account	The tiers fall from $0.50 to $0.05 per GB and reset monthly. Fragmented delivery pays the top rate in every account, permanently.
Infrequent Access	Per log group, at creation, only where nothing reads it in real time	Halves ingestion, and removes subscription filters, metric filters and Live Tail. The class cannot be changed afterwards.
CloudTrail	One copy of management events	The first copy is free; further copies are $2.00 per 100,000 events. Duplicate trails are the usual source of an unexplained CloudTrail line.

CloudTrail Lake prices retention into the ingestion

Lake is worth understanding separately because it inverts the model this post has described. One-year extendable retention is $0.75 per GB to ingest with the first year included, and $0.023 per GB per month after that. The seven-year option is $2.50 per GB for the first 5 TB a month, falling to $1 and then $0.50 at higher volumes, with all seven years included.

So Lake moves the retention decision to the front, where it is a purchase rather than a running cost. That is a genuine simplification for anyone whose retention period is fixed by regulation and will not change — and a poor fit for anyone still discovering how long they need to keep things, because the seven years are bought whether or not they are used.

Closing Thought

Logging spend has an unusual property: almost every control that improves it makes the bill worse, and almost every saving makes an investigation harder. The custom flow log format from #47 costs more than the default. The one-minute aggregation interval costs more than ten. The organisation trail costs more than the account trail it replaces. None of that is a reason not to do them, but it does mean the security argument and the cost argument are usually pulling against each other.

What makes the conversation tractable is knowing which number a proposal actually moves. Cutting retention on a young archive, switching a destination to avoid a charge that follows you there, or moving a log group to Infrequent Access when a metric filter depends on it — all three are common, all three sound like savings, and all three are answers to a question nobody checked.

The check is small. For any proposed saving: is it acting on the once-paid number or the recurring one, how old is the data it applies to, and what does it stop you doing? Three questions, and they are enough to tell a real reduction from a reorganisation of the same money.

Next in this series

Cost & Governance — the other side of the same problem: why cost allocation tags almost never cover the whole bill, what remains unallocatable no matter how disciplined the tagging, and how to report a shared-cost figure that a finance team will accept.

AWS Architecture Series #48 — What it costs to have kept everything