Business Challenge
A feed looks like a small piece of configuration and behaves like a distributed system, which is where all four of these come from.
Wait. It can take up to 10 minutes for any feed creation, update or deletion to take effect. Ten minutes is long enough to conclude the configuration is wrong, change it, and restart the clock on a second ten minutes.
Correct approachTreat feed changes like the organization policy changes in #19 — set once, then wait out the documented propagation before touching anything. A feed created as an incident response is not a tool available during that incident.
That is the documented behaviour, and it is the most important sentence about feeds. For a real-time feed, if the payload of your asset update exceeds the limit, your asset update is discarded. Not truncated, not retried, not delivered late.
Correct approachNever let a feed be the only record of a change. Pair it with the scheduled export from #23, so the feed provides latency and the export provides completeness — because the feed does not provide completeness and does not claim to.
The first message published to the Pub/Sub topic is a welcome message, and it is a plain string rather than the structured payload everything after it uses. A consumer written against the documented message type will meet a message that is not of that type, once, at the beginning.
Correct approachHandle it explicitly at the top of the consumer. It is a single line, and it is the difference between a subscriber that starts cleanly and one that crash-loops on its first delivery.
That works, and for a reason worth understanding. You can create up to 200 feeds on a parent — and the limit applies only to feeds directly on that parent, not to the feeds of its children. Ten projects each get their own 200, and the organization gets 200 of its own on top.
Correct approachRead the ceiling as per-node rather than per-estate. It is generous enough that hitting it usually means the design is wrong — one feed per resource instead of one feed per question.
Architecture
Cloud Asset Inventory is a global metadata inventory service that lets you view, search, export, monitor and analyse your Google Cloud asset metadata. Post #23 covered the first three verbs. Monitor is this one, and it is the only one that pushes.
What a feed watches
A feed monitors the same metadata the inventory holds — resources, IAM policies, access policies and organization policies — scoped to an organization, folder or project, and delivers to a Pub/Sub topic you own. That scoping matters: unlike asset history, which #23 showed is organization and project only, a feed can be attached to a folder. The level with no history endpoint does have a change stream.
Conditions, and why they are the whole design
Conditions are written in Common Expression Language, the same language the custom constraints in #20 use, and they filter which changes produce a message. An expression referencing temporal_asset.asset.ancestors restricts notifications to resources located within a particular folder, for instance.
The condition is where a feed becomes useful rather than merely noisy. An unconditioned feed on a busy organization is a firehose that nobody reads and that costs Pub/Sub throughput to ignore. The discipline is one feed per question — who made a bucket public, when did an organization policy change, which service accounts gained a role — with the condition encoding the question. That is also why 200 per parent is generous: a design with more than a handful of questions per node is usually enumerating resources instead.
Notifications arrive as a TemporalAsset, which carries the asset, the time window in which it was observed, and a prior asset state whose values include PRESENT, DOES_NOT_EXIST and DELETED. That prior state is what makes a feed actionable rather than merely informative: a message saying a resource exists is a fact, and a message saying it exists and previously did not is an event you can act on. Deletions are visible for the same reason — the asset arrives carrying its state, not as an absence you have to infer.
Who can create one
Feeds need Cloud Asset Owner. Post #23 noted the gap between that and Cloud Asset Viewer and deferred the explanation to here: Viewer reads metadata, Owner reads metadata and works with feeds. The difference is not the volume of data reachable — it is that a feed is a standing subscription that sends your estate's changes to a Pub/Sub topic of the creator's choosing, continuously, until someone deletes it.
That makes Owner an egress permission wearing an inventory name. Granting it because somebody needs to read asset metadata is over-granting; Viewer is the role for reading.
Why This Architecture Holds Up
Discarded is a different failure from delayed
Most delivery systems degrade by getting slower. A feed degrades by losing the message entirely: an asset update whose payload exceeds the limit is discarded. There is no dead-letter behaviour to inspect and no error to catch, because from the consumer's point of view nothing happened.
The bias in what gets lost is what makes this architectural rather than operational. Payload size scales with the amount of configuration attached to a resource, so the updates most likely to exceed a limit are the large, heavily-configured, much-modified ones — a sprawling IAM policy, a big instance template, a cluster with a long list of node pools. Those are also the changes most worth being notified about. The loss is not random; it is concentrated in the interesting tail.
Post #23 argued for a scheduled export to BigQuery as the durable record. This post is the reason that argument is not optional. A feed gives you seconds-to-minutes notice of a change and drops what it cannot carry; an export gives you a complete snapshot on a schedule and tells you nothing in between. They fail in opposite directions, which is precisely why running both is not redundancy. If a compliance question ever depends on "we were notified of every change", a feed alone cannot support that claim — and the documentation says so.
Pub/Sub is a dependency, not a detail
The feed publishes into your topic, which means its throughput is bounded by Pub/Sub's publisher throughput limit for the region — and that limit affects the combined asset update size a real-time feed can publish. The topic is yours, so its configuration, retention and subscription health are your responsibility, and a subscription nobody drains is a backlog rather than an alert.
This is the ordinary shape of an event pipeline, and it is worth stating because feeds are configured from the Asset Inventory side and then forgotten. The half that breaks in practice is the half you own.
What to build
- One feed per question, with the question written into the CEL condition and repeated in the feed's description. A feed whose purpose is not recoverable from its own configuration will not be deleted when the question stops mattering.
- A consumer that handles the welcome message and treats the prior asset state as the signal rather than the payload alone.
- Both a feed and the scheduled export, with the export as the source of truth for any claim about completeness.
- An alert on subscription backlog, because the feed's own health is invisible from the Asset Inventory side.
Key Architecture Decisions
| Decision | Choose this | Because |
|---|---|---|
| Feed as the record of changes | Never alone — pair with the export | An update whose payload exceeds the limit is discarded, not delayed. |
| After creating or editing a feed | Wait out the propagation | It can take up to 10 minutes for a creation, update or deletion to take effect. |
| Feeds as incident response | Create them before the incident | Ten minutes to take effect makes a feed useless as a reaction. |
| Scoping a feed | One feed per question, condition in CEL | An unconditioned feed on a busy organization is a firehose nobody reads. |
| Watching a folder | A feed works where history does not | Asset history is organization and project only; feeds are not. |
| Worrying about the feed ceiling | Usually a design smell | 200 per parent, counted per node rather than across the estate. |
| Writing the consumer | Handle the welcome message explicitly | The first message is a plain string, unlike everything after it. |
| Acting on a notification | Read the prior asset state | It turns "this exists" into "this was just created", which is the actionable form. |
| Granting feed access | Cloud Asset Viewer unless feeds are needed | Owner is the ability to stream the estate's changes to a chosen topic. |
| Operating the pipeline | Alert on subscription backlog | Feed health is not visible from the Asset Inventory side. |
Closing Thought
Feeds complete a picture the last two posts have been assembling. Asset Inventory remembers 35 days. An export remembers whatever you decide. A feed remembers nothing at all — it has no memory by design, only latency, and it trades completeness for speed in a way the documentation is unusually direct about.
That directness is worth taking at face value. A great deal of platform tooling is built on the assumption that an event stream is an authoritative log of what happened, and here it explicitly is not: an update too large to carry is dropped, and the drop is silent. Used as a trigger — notice this, then go and look — a feed is excellent, and the ten-minute setup delay and the per-parent ceiling barely matter. Used as evidence, it is the wrong instrument, and no amount of subscriber engineering fixes that. The export is the evidence. The feed is the reason you go and read it today rather than next quarter.
#25 leaves the inventory for the thing that creates most of it: Terraform on Google Cloud — the provider, where state lives, and the decisions about state that are hardest to reverse.
Comments