Home› Blog› Week 7 — Cloud Asset Inventory: The Rule I Wrote a…
GCP Weekly Lab GCP Terraform

The rule I wrote and never enforced

Two weeks ago I built a constraint saying every project must live in a folder, and left it in dry run. This week I exported the whole organization into BigQuery and asked which projects break it. Two do — one of them mine, one of them a project I had forgotten existed.

Verified against current vendor documentation on 29 September 2026. Pricing, limits and API behaviour were checked against the official docs on that date. Cloud services change fast — if you are reading this much later, treat the specifics as a starting point and re-check the linked sources.
GCP Platform Engineering Lab · Week 7 of 58

Why — The Problem This Solves

Six weeks of this lab have built a hierarchy: folders, projects, policies, a Shared VPC, a project factory, a billing export. Asked what exists in this organization right now, the only honest answer was to open the console and scroll.

Fine at seven projects, useless at seventy — and it cannot answer the question that matters: which of the rules I have written are being broken right now? Cloud Asset Inventory turns the organization into a table you can query. This week wires it up and asks it two questions.

What You Need to Know — Skills & Tools

Google Cloud

  • Cloud Asset Inventory — snapshots and feeds, at organization scope
  • BigQuery — where the snapshot lands and gets queried
  • Pub/Sub — where the feed publishes changes
  • IAM service agents — the identity Google uses on your behalf

Tooling

  • Terraform — google_cloud_asset_organization_feed
  • HCP Terraform — remote runs, Workload Identity Federation
  • gcloud asset export — the half that is not declarative
  • Standard SQL — enough for COUNT and ARRAY_LENGTH

Concepts to understand before starting

Snapshot = what IS Feed = what CHANGED Google writes as itself Creation-time rules are not retroactive

Architecture — How It Fits Together

Two shapes of the same data organization 7 projects, 4 folders 213 assets feed · Terraform export · gcloud Pub/Sub · asset-hierarchy-changes Project and Folder changes only delivered in under 25 seconds BigQuery · asset_inventory.hierarchy every asset, point in time table created by the export, not Terraform what CHANGED a stream, for reacting what IS a table, for asking Both land in the logging project, which already holds the billing export — so cost and inventory can be joined.

The feed is a Terraform resource; the export is not. That is not a provider gap — a snapshot is an operation with a moment attached, not a thing you own. Run it twice and you get two different answers, neither of which is drift.

How We Built It — Step by Step

1. The Terraform, and how it deploys

Five resources: the API on two projects, a BigQuery dataset, a Pub/Sub topic, and the feed itself. The feed is the only interesting one.

resource "google_cloud_asset_organization_feed" "hierarchy" {
  billing_project = var.seed_project_id
  org_id          = var.org_id
  feed_id         = "hierarchy-changes"
  content_type    = "RESOURCE"

  asset_types = [
    "cloudresourcemanager.googleapis.com/Project",
    "cloudresourcemanager.googleapis.com/Folder",
  ]

  feed_output_config {
    pubsub_destination { topic = google_pubsub_topic.asset_changes.id }
  }
}

Two asset types, not everything — a hierarchy change is rare and meaningful, and worth not burying under every metadata write in the org.

Deployment is three commands, because the workspace holds everything else:

terraform init      # binds this directory to HCP workspace gcp-week-07
terraform plan
terraform apply

The run executes in HCP Terraform, not on my laptop. HCP mints an OIDC token for the run, Google exchanges it through a workload identity pool that trusts app.terraform.io under an attribute condition, and the run receives a short-lived token impersonating a service account. There is no service account key, and no Google credential on the machine that typed apply. Input variables — the organization ID, the project IDs — live in the workspace, which is why there is no tfvars file in the repository.

Plan runs as a read-only identity and apply as a write-capable one, so a mistake fails before it can write. Mostly. See the challenges.

HCP Terraform run list for the Week 7 workspace showing an applied run and four errored runs above it
Nine resources applied, and four errored runs kept in frame. Those are the three permission failures below — the build was not as smooth as a cropped screenshot would suggest.

2. The snapshot, which is not Terraform

gcloud asset export \
  --organization="$ORG_ID" \
  --content-type=resource \
  --bigquery-table="projects/$PROJECT/datasets/asset_inventory/tables/hierarchy" \
  --output-bigquery-force

Asynchronous: gcloud returns an operation name and the rows appear a minute later. --output-bigquery-force overwrites rather than appends, because appending would turn the table into overlapping pictures with no column saying which is which.

The hierarchy table in BigQuery showing its schema: name, asset_type, resource, ancestors and update_time
Terraform made the dataset; the export made this table. The field that matters is ancestors — a repeated column of the resource's parents, which turns "which projects are not in a folder" into one line of SQL.

3. Asking the two questions

First: which projects sit outside a folder? A project's ancestors array is itself, then its parents, so a project directly under the organization has exactly two.

SELECT JSON_VALUE(resource.data, '$.projectId') AS project_id,
       ARRAY_LENGTH(ancestors)                  AS depth
FROM `…asset_inventory.hierarchy`
WHERE asset_type = 'cloudresourcemanager.googleapis.com/Project'

project_id                 state             depth
dev-adapter-506301-k1      ACTIVE                2   ← at the org root
katta698-gcp-lab-seed      ACTIVE                2   ← at the org root
katta698-gcp-dev-app-01    DELETE_REQUESTED      4
katta698-gcp-dev-app-02    ACTIVE                4
katta698-gcp-logging       ACTIVE                3
katta698-gcp-net-hub       ACTIVE                3
katta698-gcp-security      ACTIVE                3

In Week 5 I wrote a custom organization policy constraint requiring a project ID to start with the org prefix and its parent to be a folder. It is still in dry run. Both of these projects predate it, dev-adapter-506301-k1 breaks the naming half as well, and I had genuinely forgotten it existed.

Second: are there any service account keys? This lab's standing rule is none, ever. The naive query says two, which is alarming for about fifteen seconds:

service_account   key_type         key_origin
…719141           SYSTEM_MANAGED   GOOGLE_PROVIDED
…820355           SYSTEM_MANAGED   GOOGLE_PROVIDED

Both are Google's own rotating signing keys, not downloadable credentials. User-managed keys: zero. The inventory answered exactly what I asked, which was not quite what I meant.

The Cloud Asset Inventory console dashboard showing resource counts by type and by project
The console view of the same data — and a warning. "Overview data last updated: September 23", six days stale, next to an export that was minutes old. The counts disagree accordingly. For a question that matters, query the snapshot.

4. Testing the feed

A subscription has to exist before the change, or there is nothing to catch. Then: create a folder, wait, pull.

assetType  : cloudresourcemanager.googleapis.com/Folder
name       : //cloudresourcemanager.googleapis.com/folders/336002261512
displayName: week07-feed-test
window     : 2026-09-29T20:05:27.608Z

Under 25 seconds from folder creation to message. Deleting it produced another. The test folder is now in DELETE_REQUESTED and purges itself.

Challenges — What Actually Went Wrong

1. Google writes as itself, not as you

Three separate 403s, all the same cause: Cloud Asset Inventory delivers to every destination as its own service agent, and being project owner on that destination does not help. The agent needs pubsub.publisher on the topic, bigquery.dataEditor on the dataset and bigquery.jobUser on the project.

A shape rather than three facts: when a Google service writes somewhere for you, check what it may do, not what you may.

2. The same product, two very different errors

Creating the feed without the publisher binding names the service account, the exact permission and the resource. The identical missing-grant condition on the BigQuery destination returns The caller does not have permission and nothing else — no details array, no field.

Same service, same afternoon. When you get the useless one, the fastest route is a guess you can test in one call, not more documentation.

3. There is no read-only role that can read a feed

roles/cloudasset.viewer does not carry cloudasset.feeds.get. Only roles/cloudasset.owner does, and that also grants create, update and delete. Every HCP apply runs a plan first, so the plan identity had to be given owner or the resource could not be managed at all.

Plan is read-only everywhere else in this lab. That guarantee now has one hole, written into the CI config rather than left to be discovered.

Security — Controls at Every Layer

The audit is the control. Six weeks of guardrails were all preventive — policies refusing things at creation. This is the first detective one, and it immediately found two resources no policy would ever have caught.

The feed scope is narrow on purpose. Projects and folders only. A feed on everything is an alert that fires constantly, which is an alert nobody reads.

A human cannot borrow the CI identity. The apply service account is impersonatable only by HCP's principal set, never by a person — so running the export by hand cost me one read-only grant of my own.

Cost

$0. Cloud Asset Inventory exports and feeds are free. The snapshot is 213 rows of BigQuery storage, far below the 10 GiB free tier, and the topic has no subscriber so nothing accumulates in it.

Both destinations share a project with Week 6's billing export, so cost and inventory are one join apart — what a resource is, and what it charged.

Cleanup

This week is exempt from teardown. A feed has value only while it runs, and the snapshot is what later weeks compare against. Destroying it removes the ability to answer the question it was built for.

If you do tear it down, delete the feed before the topic. A feed pointing at a topic that no longer exists does not error; it simply stops delivering, which is the failure mode you would least like to discover by noticing nothing.

References

Not answered anywhere I looked, and measured instead: that the service agent needs three separate grants on the destinations; that no read-only role can read a feed; and that a feed delivers in under 25 seconds.

Key Takeaways

  • A creation-time rule says nothing about what already exists. Policies govern what comes next. Finding out what came before is a different tool, and you need both.
  • Check what the service is allowed to do, not what you are. Owner on the destination bought me nothing three times in one afternoon.
  • The naive query is usually the wrong query. "Two service account keys" and "zero user-managed keys" are the same data, and only one of them is the answer.
  • Query the source, not the dashboard. The console summary was six days stale while sitting next to live data.

Next week: what the hierarchy is actually doing — logs, sinks, and getting them somewhere they can be read.

Comments

How was your experience?
Your feedback helps improve this site.
PoorExcellent