Building an Account Vending Machine with AWS Organizations, SCPs & Step Functions (No Control Tower)

Why — The Problem This Solves

Every growing AWS footprint eventually hits the same wall: someone needs a new AWS account, and giving them one safely is more work than it sounds. A new team needs a sandbox. A new environment needs isolation. A security review wants every workload in its own account, not crammed into a shared one.

Done manually, every new account means a human clicking through account creation, then remembering to attach the right guardrails, then placing it in the right part of the org — and hoping nothing gets forgotten under deadline pressure. That's exactly the kind of repetitive, error-prone provisioning this lab series has been automating since Week 1, just applied one level up: instead of vending EC2 instances or databases, this week vends entire AWS accounts.

The Business Value

This project builds the same underlying pattern Control Tower's Account Factory automates — minus the permanent Landing Zone:

What You Need to Know — Skills & Tools

This week trades the data-pipeline complexity of Week 4 for organizational and IAM complexity. Before attempting it, you should be comfortable with:

Required AWS Knowledge

Required Engineering Skills

Concepts to Understand Before Starting

Architecture — How It Fits Together

Why Nest Under an Existing OU Instead of the Root?

This Organization already had real, active accounts and Control-Tower-style OUs (Core-OU, Archived-Accounts, Workloads-OU) sitting at the root — leftovers from a Landing Zone that had been enabled and later decommissioned. Rather than scatter new Sandbox/Production OUs alongside that existing structure, this project nests them inside the existing Workloads-OU, as siblings of the existing Dev OU. Always read the current state of a shared resource like an Organization before assuming it's a blank slate.

Why a Separate Lambda per Pipeline Step?

Following the same action-dispatch pattern as Week 4's glue_trigger, account_creator handles both create_account and get_account_status via an action key, since they share the same boto3 client and IAM role. account_mover and status_notifier are separate because they need genuinely different IAM permissions — least privilege is easier to enforce when a Lambda's role only ever needs to do one category of thing.

How We Built It — Step by Step

Step 1: Terraform Modules

Six Terraform modules: organizations, scp, iam, lambda, step-functions, and api-gateway. The dev environment wires them together, with the Organizations OU lookup done via a real data source rather than assumed:

data "aws_organizations_organizational_units" "root_children" {
  parent_id = data.aws_organizations_organization.current.roots[0].id
}

locals {
  workloads_ou_id = [
    for ou in data.aws_organizations_organizational_units.root_children.children :
    ou.id if ou.name == var.parent_ou_name
  ][0]
}

module "organizations" {
  source    = "../../modules/organizations"
  parent_id = local.workloads_ou_id
  ou_names  = var.ou_names   # ["Sandbox", "Production"]
}

Step 2: Service Control Policy

aws organizations enable-policy-type \
  --root-id <root-id> \
  --policy-type SERVICE_CONTROL_POLICY

# verify it took effect
aws organizations list-roots

The Sandbox OU gets a guardrail SCP denying large/expensive EC2 instance types and restricting activity to allowed regions — attached once, inherited by every account that ever lands in that OU:

resource "aws_organizations_policy" "sandbox_guardrail" {
  name = "sandbox-guardrail-scp"
  type = "SERVICE_CONTROL_POLICY"

  content = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid      = "DenyLargeInstanceTypes"
      Effect   = "Deny"
      Action   = ["ec2:RunInstances"]
      Resource = "arn:aws:ec2:*:*:instance/*"
      Condition = {
        StringLike = { "ec2:InstanceType" = var.denied_instance_types }
      }
    }]
  })
}

resource "aws_organizations_policy_attachment" "sandbox_guardrail" {
  policy_id = aws_organizations_policy.sandbox_guardrail.id
  target_id = var.sandbox_ou_id
}

Step 3: Deploying via HCP Terraform (VCS-Driven)

State lives in HCP Terraform, same as Week 5 — but this week's workspace required getting the execution model right. A CLI-driven (non-VCS) HCP workspace only uploads the directory you run terraform from, which breaks relative module sources like ../../modules/organizations that reach outside that directory. Connecting the workspace to the GitHub repo (matching Week 5's pattern) fixes it: HCP checks out the full repo, so every module path resolves, and applies are confirmed in the HCP UI run page rather than via local CLI.

The OUs now exist, nested under the existing structure rather than sitting at root:

With the apply complete, the SCP is live and inherited by anything that lands in the Sandbox OU:

Step 4: The Vending Pipeline (Lambda + Step Functions)

Account creation is asynchronous — organizations:CreateAccount returns a request ID immediately, and the real account takes 1–3 minutes to provision. Step Functions polls it with the same Wait → Check → Choice pattern from Week 4:

No live ServiceNow instance needed to test this — a small script crafts the same signed webhook payload a real ServiceNow Business Rule would send:

Challenges — What Actually Went Wrong

This week had more genuine surprises than any prior week — mostly because it touches shared org-level state instead of a clean, empty account.

Challenge 4: Step Functions Failure Path Crashed Before Notifying ServiceNow

What happened: When account creation genuinely failed, the execution status showed FAILED with zero invocations of status_notifier — meaning ServiceNow never got told. The VendingFailed state referenced $.status_result.failure_reason directly, but that path only ever gets set by the polling branch — when CreateAccount itself fails, only $.error exists, so the JSONPath never resolves and Step Functions fails the task before invoking the Lambda at all.

Fix: Added a NormalizeFailure Pass state that both CreateAccount's and MoveAccount's Catch blocks route through first, mapping $.error.Cause into $.status_result.failure_reason before reaching VendingFailed — so the same field always exists regardless of which branch failed.

Challenge 5: Hit the AWS Organizations Account Limit Mid-Test

What happened: The first real end-to-end test failed with ConstraintViolationException: You have exceeded the allowed number of AWS accounts. The Organization already had 10 accounts — 7 active, 3 already SUSPENDED from the earlier Control Tower decommission — right at AWS's default account-count limit, which counts suspended (not-yet-fully-deleted) accounts too.

Fix: Closing a few legacy active accounts didn't help immediately — closed accounts stay SUSPENDED and still count against the same limit until their ~90-day deletion window completes. A real quota increase via AWS Support is the only way to unblock new account creation today. This is itself a useful, real lesson: production account-vending systems need their quota requested ahead of the demand curve, not discovered at the moment they're needed.

Security — Controls at Every Layer

Cost

The vending pipeline itself is essentially free to run — the real cost consideration is what happens after an account is vended.

Cleanup — Tearing It Down

Because this workspace is VCS-connected, destroy has to be confirmed in the HCP UI, not via local CLI — the same restriction Week 5 ran into.

References

• AWS Organizations — Service Control Policies — how SCPs set a permission ceiling, not a grant.
• Enabling and disabling policy types on a root — the one-time root-level step this build hit (Challenge 3).
• aws_organizations_organizational_unit — Terraform resource for OUs.
• aws_organizations_policy / aws_organizations_policy_attachment — SCP creation and attachment.
• HCP Terraform CLI-driven run workflow — documents the working-directory upload limitation behind Challenge 2.
• Full source code — week-06-account-vending-machine

Takeaways — What I'd Do Differently