Week 3 — The Pin Held. The Default Moved Anyway.

Why — The Problem This Solves

Weeks 1 and 2 built governance: a landing zone with a deny, then policies that repair what they find. Both assumed somebody was creating resources correctly in the first place. This week is about that somebody — the platform team who hands other teams a module and then has to live with every decision baked into it.

Microsoft publishes Azure Verified Modules, and they are good. What they are not is yours. AVM ships a correct storage account; a platform team needs their storage account — the same resource with the arguments already decided, the tag their policy enforces already applied, and diagnostics already wired to the workspace the estate actually reads. The gap between those two is a wrapper module, and the wrapper is the easy part.

The hard part is distribution. Point ten teams at ../../modules/storage-baseline and every one of them is on whatever is in the working tree right now. A relative path has no version. So the question "which version of the storage module is the payments team running" has no answer — not a hard one, no answer — and the day you fix a default, you have changed every consumer simultaneously and told none of them.

What You Need to Know — Skills & Tools

A Terraform module is a reusable bundle of resources with inputs and outputs — you call it instead of copying its contents. A private registry is somewhere to publish your own, so other configurations reference them by name and version exactly as they would a public one. That versioning is the whole subject:

• Azure Verified Modules (AVM) are Microsoft's maintained Terraform modules. Worth knowing they are built on the AzAPI provider, which talks to Azure's REST API directly, rather than azurerm — the azurerm in the registry address is a namespace label, not a dependency. Consuming one puts azapi, modtm and random in your lock file whatever your own code uses.
• HCP Terraform's private registry can be published to by API rather than by connecting a repository, which is what lets a monorepo publish one subdirectory without splitting every module into its own repo.
• Semantic versioning is a promise about breakage. A major bump means "this will break you". The useful distinction is between a breaking change that fails loudly at plan time, and one that applies cleanly and changes behaviour at runtime — the second is the dangerous kind.
• A version pin is only as good as what can change it. A module's version must be a literal: not a variable, not a -var flag. Upgrading is therefore a code change and a commit, never a runtime switch — which is precisely the property that makes pinning worth anything.

Architecture — How It Fits Together

Three pieces. A wrapper module in the lab's monorepo. A publish script that turns a git tag into a registry version. And a single Terraform configuration holding two consumers of that module, pinned to two different majors — which is not a contrived arrangement, it is the permanent condition of any platform team that owns a module and has no authority to make everyone upgrade on the same day.

The wrapper pins AVM exactly, and that is deliberate asymmetry

module "storage_account" {
  source  = "Azure/avm-res-storage-storageaccount/azurerm"
  version = "0.10.0"          # EXACT, not ~> 0.10

  location  = var.location
  name      = local.account_name
  parent_id = var.resource_group_id    # AVM takes the full RG id, not a name

  https_traffic_only_enabled      = true
  min_tls_version                 = "TLS1_2"
  allow_nested_items_to_be_public = false
  shared_access_key_enabled       = var.shared_access_key_enabled

  diagnostic_settings_storage_account = local.account_diagnostics
  diagnostic_settings_blob            = local.blob_diagnostics
}

The wrapper pins Azure/avm-res-storage-storageaccount/azurerm at exactly 0.10.0 — not ~> 0.10. That is a different decision from the one a consumer of the wrapper makes, and the reason is that a wrapper's whole job is to be the stable thing. If it floated, a caller pinned to storage-baseline 1.0.0 could still have their storage account change underneath them, which is the exact outcome pinning was supposed to prevent. AVM is also pre-1.0, where semver permits a minor bump to break.

One workspace ID in, two diagnostic settings out

A storage account emits no logs of its own — only the Transaction metric. The read, write and delete audit events belong to the blob service, a separate ARM resource at .../blobServices/default with its own diagnostic setting. Asking for allLogs on the account is accepted and produces nothing, which is the sort of thing discovered during an incident rather than during a deploy. The wrapper takes one workspace ID and creates both settings, so no caller has to know that.

How We Built It — Step by Step

0. How any of this gets deployed

Terraform throughout. State lives in HCP Terraform and the apply runs locally — HCP defaults a new workspace to remote execution, which fails here because no Azure credentials are stored in the workspace.

Nothing below is typed into a terminal by hand. The whole week is six commands, and the order matters more than it looks:

./scripts/publish.sh 1.0.0     # BOTH versions must exist before either stage runs
./scripts/publish.sh 2.0.0
./scripts/deploy.sh v1         # one consumer, pinned to 1.0.0
./scripts/deploy.sh v2         # a second on 2.0.0, alongside the first
./scripts/validate.sh
./scripts/cleanup.sh

count = 0 does not mean "not downloaded". terraform init resolves and installs every module a configuration refers to before it evaluates what any count is — so 2.0.0 has to exist in the registry before stage v1 can initialize at all. That is why both publishes come first.

terraform {
  cloud {
    organization = "Katta"
    workspaces {
      name    = "azure-week-03-dev"
      project = "Azure Platform Lab"
    }
  }
}

The registry itself, read back from the HCP Terraform API rather than described — any reader with a token for their own organization can run the same call and see the same shape:

GET /api/v2/organizations/Katta/registry-modules/private/Katta/storage-baseline/azurerm

module: storage-baseline / azurerm   status: setup_complete
   version 2.0.0 -> ok
   version 1.0.0 -> ok

Two versions, both published, both ok. That is the factory — and this is it in the private registry:

1. The module, published from a tag

publish.sh builds the tarball with git archive <tag>:<path> — never from the working tree. A registry version is immutable, so a tarball built from a dirty checkout publishes a version matching no commit, and the only correction available is deleting a version other people may already be pinned to.

2. Two versions, one breaking change of each kind

2.0.0 carries exactly two differences from 1.0.0, chosen to be opposites:

3. Two consumers, deployed in stages

Stage one creates app_a on 1.0.0. Stage two adds app_b on 2.0.0 beside it and never touches the first block again.

# Pinned to 1.0.0 and never touched again. Passes no workspace, because
# 1.0.0 allowed that - this block is the evidence of what the defaults were
# on the day it was written.
module "app_a" {
  source  = "app.terraform.io/Katta/storage-baseline/azurerm"
  version = "1.0.0"

  workload          = "appa"
  resource_group_id = azurerm_resource_group.week.id
  cost_center       = "platform-lab"
}

# Added later, alongside it. An exact pin, not ~> 2.0: a range is a standing
# agreement to skip reading the diff, and the diff is the safety mechanism.
module "app_b" {
  source  = "app.terraform.io/Katta/storage-baseline/azurerm"
  version = "2.0.0"
  count   = var.enable_v2_consumer ? 1 : 0

  workload                   = "appb"
  resource_group_id          = azurerm_resource_group.week.id
  cost_center                = "platform-lab"
  log_analytics_workspace_id = azurerm_log_analytics_workspace.diagnostics.id
}

The plan for stage two is the evidence, and it is a claim about a plan rather than about a deployment — so the script saves the plan, reports on it, and only then applies:

create   module.app_b[0]...azapi_resource.this
create   module.app_b[0]...containers["data"]
create   module.app_b[0]...diagnostic_setting_blob
create   module.app_b[0]...diagnostic_setting_storage_account

module.app_a changes in this plan: 0

Four resources created, every one of them app_b. A new major landed in the same state file and the consumer pinned to the old one did not move. That is what the pin buys.

Challenges — What Actually Went Wrong

Both breaking changes are two lines of the module's own variables:

# 1.0.0                          # 2.0.0
variable "log_analytics_workspace_id" {
  type    = string               type = string
  default = null                 # no default - now REQUIRED  (the LOUD break)
}

variable "shared_access_key_enabled" {
  type    = bool                 type = bool
  default = true                 default = false            (the SILENT break)
}

The silent break is only visible on deployed resources

Both accounts came from the same module, through the same code path, and neither call site mentions shared_access_key_enabled at all. Here is 1.0.0:

And 2.0.0, with every other setting on the blade identical:

Nothing in either configuration says so. No plan flagged it. This is the half of a major bump that a range constraint hides completely — and a team that writes ~> 2.0 after upgrading has re-armed it for 2.1.0.

An empty module published, and every signal said success

The first publish uploaded a valid 45-byte tarball containing nothing. git archive <tag>:<path> resolves the path relative to the current directory, and the script ran from the week directory with a repo-root-relative path — so it did not error. It resolved to an empty tree. The upload returned 200, the version reached status ok, and a consumer would have met the result as a module with no inputs rather than as a failure.

Worse, the obvious guard does not work: git rev-parse -q --verify "<tag>^{tree}:<path>" still returns a hash for the missed path. The only honest check is counting the files in the tarball before uploading, which the script now does.

A pending version blocks its own retry

That failed publish created the 1.0.0 version record before it broke. The version then sat at status pending, and the retry was refused — "already exists, registry versions are immutable". A version awaiting an upload it will never receive has to be deleted through the API before republishing. Safe here only because nothing was pinned to it yet, which will not be true the second time this happens.

AVM nests deeply enough to hit Windows MAX_PATH

The AVM storage module pulls avm-utl-interfaces once per sub-resource, producing paths like .terraform/modules/app_a.storage_account.queues.role_assignments.interfaces/.git/objects/pack/pack-<sha>.pack. The clone fails with Filename too long — and it fails with core.longpaths=true set and the Windows LongPathsEnabled registry value already 0x1. The module cache is relocatable and the configuration is not, so the fix is to move TF_DATA_DIR somewhere short rather than to move the repository.

A validation that failed on correct resources

Two checks reported failures against resources that were perfectly correct. az renders JSON booleans Python-style — True and False, capitalised — and the script compared against lowercase "false". It is the same shape as a check that passes because the lookup broke, inverted: a check that fails because the comparison did.

Security — Controls at Every Layer

• The baseline is not optional. TLS 1.2 minimum, HTTPS-only transport, and anonymous blob access disabled are decided in the module, not passed in by callers. A caller cannot forget them because there is no argument to forget.
• 2.0.0 turns shared keys off by default — the flipped default that makes this week's point also happens to be the more secure posture. A caller who genuinely needs keys can still ask for them, explicitly, in their own code rather than inheriting it.
• The tag the estate enforces is applied at creation. Every account carries cost-center, so week 2's remediation policy never has to touch anything this module made.
• Diagnostics reach a workspace — account metrics and blob service audit logs, and in 2.0.0 there is no way to opt out by omission.
• One honest exception: public_network_access_enabled is true. The alternative is a private endpoint, and a private endpoint without a linked private DNS zone resolves to the public IP and fails in a way that looks like a firewall problem. The zone estate is week 5; the module moves when it exists.

Cost

Two Standard LRS storage accounts and one Log Analytics workspace at PerGB2018 with 30-day retention. Nothing in this week has a per-hour price — no compute, no gateway, no firewall — so the bill is driven entirely by data, and this week stored and ingested essentially none. Pennies while running.

Cleanup

Two kinds of thing were created and only one of them should go. The consumers — a resource group, a workspace, two storage accounts — cost money and prove nothing further, so they are deleted.

The registry versions are kept. They cost nothing, they are the week's actual output, and deleting a published version is the one destructive act a module registry does not forgive: every configuration pinned to it stops initialising, including configurations owned by people who were never asked. The cleanup script takes a --registry flag to remove them and does not do it by default.

References

• Azure storage account overview
• Prevent Shared Key authorization for an Azure Storage account
• Monitor Azure Blob Storage
• Azure Verified Modules
• Week 3 code in the lab repository

Key Takeaways

• A relative module path has no version. Every consumer is on HEAD, and "which version is this team running" has no answer. That is the problem a private registry solves, and it is not really about convenience.
• The pin protects the caller who does nothing. A new major landed in the same state file and the 1.0.0 consumer's plan showed zero changes.
• The dangerous breaking change is the one that does not break anything. A required input fails loudly at plan and nobody gets hurt. A flipped default applies cleanly and surfaces in production, in whatever was still using the old behaviour.
• Pin exactly in the module, and exactly in the caller. A range is a standing agreement to skip reading the diff, and the diff is the actual safety mechanism.
• An immutable registry is unforgiving in both directions. It will not let you fix a bad version, and it will not let you retry a failed publish without deleting what the failure left behind.