Home› Blog› Week 2 — Keyless CI on Google Cloud: The Shortcut …
GCP Weekly Lab GCP Terraform

Keyless CI, and the shortcut that was already gone

Build a pipeline that proves who it is to Google Cloud without storing a credential — and find out what the platform has already decided on your behalf.

Verified against current vendor documentation on 26 August 2026. Pricing, limits and API behaviour were checked against the official docs on that date. Cloud services change fast — if you are reading this much later, treat the specifics as a starting point and re-check the linked sources.
GCP Platform Engineering Lab · Week 2 of 58

Lab Overview

The problem. Week 1 built a resource hierarchy by running terraform apply on a laptop. That works exactly once, for exactly one person, and leaves no record of what was proposed before it happened. The conventional fix is to give CI a service account key: a JSON file holding a private key, pasted into a pipeline variable. It never expires, it lives in a file, and it gets copied wherever is convenient.

Objective. Give the pipeline an identity with no stored credential at all — a workload identity pool trusting HCP Terraform's OIDC issuer, two service accounts split by run phase, and an attribute condition that decides who the issuer actually speaks for.

What reading the platform first changed. Listing the organization policy before writing any found seven constraints already enforced — Google's security baseline, applied automatically to every organization created on or after 3 May 2024. Two of them mean a key cannot be created and cannot be uploaded, so the planned opening move of the week was already done.

DifficultyIntermediate — the resources are few; the trust boundary is subtle and easy to leave wide open
DeploymentTerraform, hand-run on purpose and permanently — this is the layer that defines who CI is
Reproduce itLab guide — the attribute condition, the phase split, the denial
Cost$0 — pools, providers, service accounts, bindings and token exchanges are all free

Cloud Services and Concepts

ServicePurpose hereWhat it is
Workload Identity FederationLets CI act as a service account with no keyLets an identity from outside Google Cloud act as a service account, without a key. A pool holds external identities; a provider inside it names the issuer they come from.
Security Token ServicePerforms the token exchange on every runThe endpoint that exchanges an external OIDC token for a short-lived Google federated token. Where the attribute condition is evaluated.
IAM service accountsThe two identities the pipeline assumesTwo of them, tf-plan and tf-apply, holding read and write roles respectively.
Organization PolicyRead, not written - it is why no key can existNot written this week — read. The baseline it already enforces is what makes a key impossible.
Project liensStops the seed project being deleted by anyoneDeletion refused at the API, whoever asks, until the lien is deliberately removed.
HCP TerraformRuns the plans and applies, storing no credentialRuns the plans and applies, mints one OIDC token per run phase, and stores no GCP credential.

The concept that reframes everything: the issuer is public. https://app.terraform.io signs tokens for every HCP Terraform customer in the world. Configuring it as a trusted issuer does not trust your runs — it trusts all of them. The security control is not the pool, the provider, or the issuer URL. It is the CEL expression that inspects the token's claims and decides which of those customers you meant.

Impersonation is not authentication. The federated token is not the service account. It is permission to ask to be one, and the service account's own IAM policy decides whether the request is granted. That is where the plan/apply split is enforced: each account grants roles/iam.workloadIdentityUser to a principalSet scoped to one value of the terraform_run_phase claim. Google refuses the mismatch at exchange time, so it is a boundary rather than a convention.

Architecture and Workflow

No credential is stored on either side. Each run mints a token, exchanges it, uses it, and lets it expire.

The issuer is public GCP Lab · Week 2 HCP Terraform mints one OIDC token per run the boundary attribute condition organization name project name workspace prefix Icon_24px_IAM_Color STS exchanges the token tf-plan read-only roles tf-apply write roles signs phase = apply phase = plan app.terraform.io signs tokens for EVERY customer no key exists — org policy refuses to create one jayanthkatta.com hcp:80,196,268,136;cond:500,170,380,190;plan:330,540,300,126;apply:760,540,300,126

The flow

  1. Read what the organization already enforces, before designing anything.
  2. Create the pool and a provider trusting https://app.terraform.io, mapping five claims to attributes.
  3. Write the attribute condition — the one piece that turns a public issuer into a specific trust relationship.
  4. Create tf-plan and tf-apply, and bind each to a phase-scoped principalSet.
  5. Grant each identity the narrowest roles that work, individually rather than by basic role.
  6. Place a lien on the seed project, so deletion is refused at the API.
  7. Prove a plan really ran remotely, then attempt the shortcut and confirm it is refused.

The decision worth naming

This week is deliberately human-run, permanently. For tf-apply to apply this configuration it would need resourcemanager.organizations.setIamPolicy — which would let it grant itself anything — and admin on the pool, which would let it rewrite the attribute condition and trust whoever it liked. An identity must not manage its own grants. So the lab has two layers: this week and the bootstrap are hand-run and define who CI is; every other week runs remotely as CI.

Prerequisites

  • An organization with a seed project for bootstrap concerns — Week 1 supplies it. The pool, provider and both identities live there.
  • An HCP Terraform organization and workspace. Note that the organization's real name may differ in case from every screen that displays it; see Troubleshooting.
  • roles/resourcemanager.organizationAdmin on a human operator. This week cannot be run by CI, by design.
  • APIs: iam, sts and cloudresourcemanager.
  • Terraform 1.9+ with hashicorp/google ~> 6.0.

Variables, role lists and commands are in the lab guide.

Implementation

Eighteen resources at deploy time, all free. The file carries thirty-four today, because later weeks added the roles CI needed — this is the layer where CI's permissions are defined, so it grows by design. Commands are in the lab guide.

Repository structure for week 02: LAB.md, terraform, scripts and docs, each with a one-line responsibility.
How the week is organised. Full source on GitHub.

Step 1 — Read the organization policy before writing any

Why: because what you are about to build may already exist. The gcloud org-policies list call took two seconds and removed a planned section of this week.

Checkpoint: seven constraints enforced, all sharing an updateTime from the moment the organization was created.

Organization policies page showing baseline constraints including disableServiceAccountKeyCreation.
Validates Step 1: the baseline, including the two constraints that make a service account key impossible. Nobody in this lab set them.

Step 2 — The pool and the provider

Why: a pool holds external identities; a provider declares which issuer they come from and maps the token's claims to attributes that IAM can reason about.

attribute_mapping = {
  "google.subject"                        = "assertion.sub"
  "attribute.terraform_organization_name" = "assertion.terraform_organization_name"
  "attribute.terraform_project_name"      = "assertion.terraform_project_name"
  "attribute.terraform_workspace_name"    = "assertion.terraform_workspace_name"
  "attribute.terraform_run_phase"         = "assertion.terraform_run_phase"
}
The hcp-terraform workload identity pool in the console, with its provider listed.
Validates Step 2: the pool and its single provider. At this point the configuration trusts every HCP Terraform customer on earth.

Step 3 — The attribute condition, which is the actual control

Why: without a condition, the only thing between any HCP Terraform user and this organization is that they have not guessed the provider's resource name — which is not a secret and is printed in plan output.

attribute_condition = join(" && ", [
  "assertion.terraform_organization_name == \"katta\"",
  "assertion.terraform_project_name == \"GCP Platform Lab\"",
  "assertion.terraform_workspace_name.startsWith(\"gcp-\")",
])

Three claims, narrowing in turn. The third is not decoration: other workspaces of mine live in the same HCP organization and carry tokens from the same issuer. The project and prefix checks are what keep them out of this organization.

Workload identity provider configuration showing attribute mappings and the CEL condition, alongside a usage chart of successful token exchanges.
Validates Step 3: the condition in the console. The CEL box is scrolled, so the organization check sits above the fold. The panel on the right — count of successful token exchanges — is the proof the flow works.

Step 4 — Two identities, split by run phase

Why: HCP issues a token per phase, carrying a terraform_run_phase claim, so the split can be bound in IAM rather than trusted as convention.

resource "google_service_account_iam_member" "plan_federation" {
  service_account_id = google_service_account.plan.name
  role               = "roles/iam.workloadIdentityUser"
  member             = "principalSet://iam.googleapis.com/${pool}/attribute.terraform_run_phase/plan"
}
Google Cloud service accounts page showing tf-plan and tf-apply, both enabled, Key ID column reading No keys.
Validates Step 4: both identities, and the column that carries the whole argument — Key ID: No keys. Not "we deleted them". None was ever possible.

Step 5 — Prove the plan ran remotely

Why: a workspace can be set to remote while the plan quietly executes locally against your own credentials, and the output looks identical. The tell is the first line — "Running plan in HCP Terraform".

HCP Terraform run detail showing a plan that finished with no changes, triggered from CLI.
Validates Step 5: a plan running in HCP as tf-plan, refreshing Week 1's hierarchy through federation and finishing with no changes.

Step 6 — Attempt the shortcut

Action: ask for a service account key, as an identity that holds iam.serviceAccountKeys.create.

Expected: a refusal that names the policy, and no key afterwards.

Terminal output showing key creation refused with FAILED_PRECONDITION and CUSTOM_ORG_POLICY_VIOLATION, followed by an empty key list.
Validates Step 6 — the figure the post rests on: FAILED_PRECONDITION: Operation denied by org policy, naming constraints/iam.managed.disableServiceAccountKeyCreation and reason: CUSTOM_ORG_POLICY_VIOLATION. The following keys list --managed-by=user returns nothing.

Validation and Evidence

TestExpectedActualStatus
Infrastructure deployed18 resources18 added, 0 changed, 0 destroyedPass
What the baseline already enforcesread before designing7 constraints, from organization creationFinding
Workload identity poolexists, correct issuerhttps://app.terraform.ioPass
Provider attribute conditionpins organization, project and workspace prefixpresent, all three clausesPass
Service accountstf-plan and tf-apply, no keysboth; Key ID: No keysPass
Who may impersonate the apply accountthe apply-phase principalSet onlycorrectPass
Plan identity cannot writeno write rolesfive read roles, no write rolePass
The plan actually ran remotely"Running plan in HCP Terraform"present; run visible in HCP as tf-planPass
The shortcut, attemptedkey creation refused, no key createdFAILED_PRECONDITION, CUSTOM_ORG_POLICY_VIOLATION; keys list emptyPass

The last row was unverified for two months. The figure proving the week's central claim was the one missing from the original capture. It was attempted and captured on 25 October 2026, as an identity holding the permission — the only way the result means anything.

It is the MANAGED generation doing the work. The legacy constraint of the same name carries no policy and reads enforce: false. Both generations have to be checked before concluding anything about either — an absent policy reads identically to a disabled one.

A test must not be able to create the thing it disproves. The obvious way to write this check is to send the key to /dev/null. That discards the key material — but if the constraint ever failed to hold, the key would still exist server-side, on the identity that can change the estate, created by the test meant to prove none could exist. The failure branch has to clean up after itself, or the check is a liability on exactly the day it finally fires.

Security and Cost

Security decisions

ControlDecision and trade-off
No key, enforced not intendedOrganization policy refuses creation and upload. Discipline is not a control; a denial from the API is.
The attribute condition is the trust boundaryA public issuer plus no condition trusts every customer of that issuer. Three claims narrow it to one project's workspaces.
Plan and apply are separate identitiesBound to different run phases, so a pull-request plan cannot reach a role that writes. Enforced by Google at exchange, not by convention.
Predefined roles, not basicroles/viewer confers object read on every bucket. Replaced with five specific read roles — read-only is not the same as harmless.
Deletion protection at the APIA lien, not only prevent_destroy. Terraform's guard protects one configuration; the lien protects the resource.
Billing IAM deliberately out of scopeThe apply identity spends against a billing account; it does not manage roles on it. That grant is made once, by a human, out of band.
Accepted gapThe pool trusts one HCP organization and a workspace prefix, not one workspace. Any workspace matching the prefix can obtain a token.

Cost

$0. Workload identity pools, providers, service accounts, IAM bindings and liens are free, token exchanges are free, and HCP Terraform's free tier covers the runs. Which is the uncomfortable part: the keyless path costs nothing, takes an afternoon, and is still not what most tutorials teach.

Troubleshooting and Lessons

The given credential is rejected by the attribute condition

Problem: every token exchange refused, with an error naming no claim. Both ends looked correct.

Root cause: the HCP organization's real name is lowercase katta. Every human-facing surface shows it capitalised — the console URL, the cloud {} block, the workspace list — because HCP resolves those case-insensitively. The OIDC token carries the true value, and Google's CEL comparison is case-sensitive.

Lesson: when a comparison is case-sensitive on one side and not the other, the displayed value is not evidence. Cost roughly forty minutes.

A read-only role that could read every object in the estate

Problem: the plan identity was given roles/viewer at the organization, on the reasoning that a plan only reads, so breadth costs nothing.

Root cause: documented, and worth reading before granting. roles/viewer confers roles/storage.legacyObjectReader on buckets. So it was not "read the hierarchy" — it was read every object in every project in the organization, including projects that did not exist yet.

Resolution: five specific read roles.

Cleanup and Key Takeaways

Cleanup

None, deliberately. This is permanent infrastructure — every later week runs through it, and tearing it down would take the pipeline with it.

One thing resists cleanup on purpose: the lien on the seed project. Removing it is a separate, explicit act requiring resourcemanager.projects.updateLiens. That is the point of it.

Ongoing charges: none. Nothing here bills per hour.

Key takeaways

  1. Read the organization policy before you write any. A two-second command removed a planned section of this build. Organizations created on or after 3 May 2024 already refuse service account keys — and if yours predates that, nothing turned the constraints on for you.
  2. The issuer is public; the claims are the boundary. Trusting app.terraform.io without an attribute condition trusts every HCP Terraform customer alive. That CEL expression is the security control, not the configuration around it.
  3. A denial must be attempted by an identity that holds the permission. An under-privileged attempt fails with a permission error indistinguishable from the control working. FAILED_PRECONDITION rather than PERMISSION_DENIED is what makes the evidence mean something.
  4. Read-only is not harmless, and a Terraform guard is not a cloud guard. roles/viewer reads object data; prevent_destroy protects one configuration while a lien protects the resource.
  5. Check where your plan actually ran. A remote workspace can still be planned locally, and the output looks the same. "Running plan in HCP Terraform" is the only proof the CI identity was involved at all.

What production would do differently

Condition on the workspace rather than a prefix, so one compromised workspace cannot assume the apply identity. Scope the write roles below the organization — serviceAccountAdmin at organization scope permits setting IAM policy on any service account, including the apply identity itself, which is a route to widening who may impersonate it. Use a variable set rather than per-workspace variables. Separate the identity per environment. And alert on token exchanges: a failed exchange is either a misconfiguration or somebody else trying, and both are worth knowing about.

References

Comments

How was your experience?
Your feedback helps improve this site.
PoorExcellent