Business Challenge
All four of these come from the same gap: the console and the CLI both present a completed action, and the API underneath presents a job number.
Both facts are true. If you make a request that mutates data, Compute Engine returns an Operation object that you then poll to get the status of your request. The 200 acknowledges the instruction; it says nothing about the outcome.
Correct approach
Treat a mutating call as accepted rather than done, and poll the Operation until it reports DONE. Every client library that feels synchronous is doing this for you.
There is a purpose-built alternative. You can get the status of an operation using the get or the wait method, and a wait request returns when the operation is DONE or when the request is approaching the 2 minute deadline. One call that blocks beats sixty that do not.
Use wait in a loop rather than get on a timer. Post #15's rate quotas and post #16's advice about short polling are the same lesson arriving from the other direction.
Almost always the hidden local state. The gcloud CLI properties are settings that affect the behavior of the tools, and a configuration is a set of those properties — a profile. A command that omits --project is not project-agnostic; it is reading a value CI does not have.
Pass the scope explicitly in anything automated. A command whose meaning depends on the machine it runs on is not a script, it is a note to yourself.
Read what it actually promises. It disables all interactive prompts, and if input is required, defaults will be used, or an error will be raised. So the prompt you were avoiding may have been answered on your behalf rather than skipped.
Correct approachUse it, and know which prompts existed. A confirmation that becomes a default acceptance is the whole risk, and it is invisible in the log.
Architecture
There is one control plane and three faces on it. The console, the CLI and the client libraries all speak to the same REST API, and each adds a layer of convenience that is easy to mistake for the platform's behaviour.
Everything that changes something returns a receipt
This is the single most useful fact about the platform's shape. A mutating request does not return the thing you asked for — Compute Engine returns an Operation object that you then poll to get the status of your request. Operations are global, regional or zonal, matching the resource they act on.
Once you know that, three things stop being mysterious. The console's spinner is a poll loop. gcloud appearing to block is a poll loop. A client library's synchronous-looking call is a poll loop. None of them are doing anything the API offers directly; they are all doing the same work, and when you write against the API you inherit it.
| Polling method | Behaviour | Use it when |
|---|---|---|
get |
Returns the Operation immediately, whatever its state. | You want a status now and will decide what to do about it. |
wait |
Returns when the operation is DONE, or when the request is approaching the 2 minute deadline. |
You are waiting for completion — which is nearly always. |
The two-minute figure is the number to design around. wait is not a promise of completion; it is a bounded block. A create that takes five minutes needs three wait calls, and a loop around wait is the correct shape rather than a workaround.
Post #16 listed Google's own guidance for reducing pressure on rate quotas: wait for operations to be done, minimize client-side retries, avoid short polling. From the API's side that guidance has a specific implementation, and it is wait rather than get. A script polling get every second burns sixty calls a minute against the quota to learn nothing sixty times; the same minute is one wait call. The rate limit and the polling method are the same conversation.
gcloud is a program with memory
The CLI is not a thin wrapper. It holds local state, and that state changes what commands mean. Properties are settings that affect the behavior of the tools; a configuration is a set of properties, working like a profile.
That is genuinely useful at a terminal and a hazard everywhere else. The same command, byte for byte, creates a resource in different projects on two machines. Nothing in the command records which. And unlike an environment variable, the value lives in a file most people have never opened.
--impersonate-service-account means that for that invocation, all API requests will be made as the given service account, or as the target account in a delegation chain, instead of the currently selected account. It is the right tool — better than a downloaded key, which post #22's baseline blocks by default. But note the shape: the audit log records the service account, the shell history records the flag, and the two live in different places. When reconstructing who did what, the flag is the part that goes missing.
Why This Architecture Holds Up
One flag replaces a lot of guessing
--log-http logs all HTTP server requests and responses to stderr. That single flag turns every question in this post into an observation: which endpoint a command hits, how many calls one invocation makes, what the request body actually contained, and which of your arguments became defaults.
It is the fastest way to learn an unfamiliar corner of the API, and the fastest way to answer "why did that take so long" — usually because one command was several calls and a poll loop. Use it on a scratch project, read the output once, and most of the platform's shape stops being folklore.
Which face to use, and when
| Face | Best at | What it hides |
|---|---|---|
| Console | Discovery, and reading state you did not create. | The calls it made, the defaults it chose, and the polling it did. |
| gcloud | Operating, scripting, and learning the API with --log-http. |
Local configuration, so the same command is not the same command. |
| REST or a client library | Anything a program owns. | Nothing — which is the point, and the cost. |
The honest ranking for anything that runs unattended is the reverse of the ranking for anything a person does once. The console is the best place to understand a service and the worst place to configure one, because it leaves no artefact — a resource created by clicking has no record of what was chosen, which is exactly the reproducibility problem the last three posts were solving.
Every default the console picks is a decision somebody has to make, and picking a reasonable one is a service to a person who does not yet know the field exists. The same helpfulness is a liability in automation, where an unspecified field should be a question rather than an assumption. That is the whole tension, and it explains why a console-built resource is so hard to reproduce: the console answered questions you were never shown.
What to carry into anything automated
- Poll, do not assume. A mutating call returns an Operation. Success on the call is not success on the work.
- Use
wait, in a loop. It returns onDONEor near the two-minute deadline, so the loop is the design, not a patch. - Pass scope explicitly. Project, region and zone on the command, never from a configuration file the CI runner does not have.
- Know what
--quietanswered. Defaults will be used where input was required. - Prefer impersonation to keys, and record the flag somewhere the audit log's service-account entry can be joined to it.
Key Architecture Decisions
| Decision | Choose this | Because |
|---|---|---|
| Handling a mutating call | Poll the Operation to DONE |
The call returns an Operation object, not the resource. |
| Polling method | wait in a loop, not get on a timer |
wait returns on DONE or near the 2 minute deadline; get returns immediately regardless. |
| Rate quota pressure from polling | Fix the polling method first | It is the concrete form of post #16's "avoid short polling". |
| Scope in automation | Explicit flags every time | A configuration is a set of properties on that machine, and CI does not have it. |
--quiet in a pipeline |
Use it, and enumerate the prompts | If input is required, defaults will be used rather than the command failing. |
| Identity for a one-off task | --impersonate-service-account |
All API requests are made as that account, with no key to leak. |
| Learning an unfamiliar API | --log-http on a scratch project |
It logs all HTTP requests and responses to stderr, including the calls you did not know about. |
| Diagnosing a slow command | --log-http before profiling anything |
One command is usually several calls plus a poll loop. |
| Creating anything durable | Not in the console | It leaves no artefact of what was chosen, which is the reproducibility problem #25 to #27 exist to solve. |
| Reading state you did not create | The console, freely | Discovery is what it is best at, and reading changes nothing. |
Closing Thought
The API underneath Google Cloud is smaller and blunter than any of its faces suggest. It does not wait. It does not retry. It does not know which project you meant. Every one of those courtesies is added by something above it, and the reason that matters is that the courtesies are invisible until you step outside them — at which point they arrive as bugs. The 200 that created nothing, the script that polls sixty times a minute, the command that works everywhere except CI.
Which suggests one habit worth more than the rest of this post. Run something familiar with --log-http and read what comes back. Not to learn the endpoints, but to see how much is being done for you by the thing you were treating as a thin layer. It is a short exercise, and afterwards the platform's odder behaviours mostly stop being odd — they were the API showing through.
#29 steps up from mechanism to a complete worked design: the enterprise foundations blueprint, section by section — what Google actually recommends for a whole organisation, and which parts of it the last twenty-eight posts have already argued about.
Comments