Homeβ€Ί Blogβ€Ί Week 23 - Service Networking with VPC Lattice: Who…
AWS Weekly Lab AWS Terraform

Who may call this service

Two Lambda functions in the same subnet behind the same security group, running identical code and differing only in their IAM role. One may call the service; the other may not β€” and one optional Terraform argument silently turns the control off.

Verified against current vendor documentation on 18 October 2026. Pricing, limits and API behaviour were checked against the official docs on that date. Cloud services change fast — if you are reading this much later, treat the specifics as a starting point and re-check the linked sources.
AWS Platform Engineering Lab Β· Week 23 of 52

Lab Overview

Your orders app calls your rates app. How do you control who is allowed to make that call?

The problem. Almost everyone answers with network rules: a security group, a CIDR range, "anything in this subnet can reach port 8080 over there". That controls addresses, not identities. Anything that lands in that subnet can make the call, and the rule cannot tell your orders app from a compromised test box beside it. You also cannot write the sentence you actually mean β€” "only the orders service may call rates" is not expressible in a security group.

The objective. Build one service that only a named IAM role may call, then try to call it as the wrong one. Two Lambda functions, same VPC, same subnet, same security group, byte-identical code. The only difference is which role signs the request.

The outcome. The control works β€” and a single optional Terraform argument silently disables it while the console still shows the policy attached and the service healthy.

DifficultyIntermediate
Lab runtimeRan 41 hours before teardown (40 billed service-hours)
Measured cost$1.00 for 40 service-hours
DeploymentTerraform via HCP Terraform with OIDC
Full lab guideGitHub β€” prerequisites, Terraform, commands, validation, cleanup

Not App Mesh. App Mesh shut down on 30 September 2026 and cannot be created. VPC Lattice is where AWS points instead, and that is the only reason App Mesh appears here at all.

Cloud Services and Concepts

ServicePurpose hereWhat it is
Amazon VPC LatticeA managed switchboard between your own applications. Each VPC attaches to a shared service network and each app registers as a service, replacing a web of peering connections and security-group rules nobody can audit.An application-layer network that connects services to each other and authenticates every call, instead of relying on subnet and security group reachability.
Service networkThe switchboard itself β€” a named meeting point that VPCs attach to and services register with. Attaching is free and removes the need for peering. It is also the first place an access policy can be applied.The Lattice object services join in order to reach one another. It authenticates callers, but on its own it does not decide which caller is allowed to do what.
Lattice serviceOne callable application on that switchboard. Not your code β€” the object that fronts it, with a stable DNS name resolvable only inside an attached VPC.A single application registered on the service network, with its own listener, routes and targets.
Auth policyAn IAM policy document attached to a service or service network naming which identities may call it. Evaluated by AWS before the request reaches your code.A resource policy attached to a Lattice service network or service that says which signed principals may call it. Without one, authentication proves who called but nothing refuses them.
AWS LambdaThe two callers and the backend. Identical code; only the execution role differs.Runs a piece of code on demand with no server to manage. You are billed per request and per millisecond it runs, and each in-flight request occupies one execution environment.
HCP TerraformRuns Terraform remotely and assumes an AWS role by OIDC, so no static keys exist anywhere.HashiCorp's hosted Terraform. It runs plans and applies on its own machines and keeps the state file, and it assumes an AWS role by OIDC so no static keys exist anywhere.

Two concepts decide the whole week, and they are easy to conflate.

auth_type is a switch set independently on the service and on the service network. AWS_IAM means evaluate the policy; NONE means skip that layer entirely. The policy and the switch are different objects and they can disagree.

SigV4 signing is how a caller proves who it is β€” the same signing the AWS CLI performs on every command. A request without a signature is not rejected for being malformed; it is treated as coming from anonymous, which a policy can allow or refuse like any other principal.

Architecture and Workflow

aws AWS Cloud Callers β€” identical code, same SG client Lambda role: orders-client intruder Lambda role: stranger 1 2 Service network authType: AWS_IAM both callers pass here: same VPC, both signed service "rates" the thing being reached 3 The service auth policy allows orders-client refuses stranger β€” 403 Delete it and both get in. 4
The network lets both callers through. Only the service's own auth policy tells them apart.
  1. Two Lambda functions run identical code in the same subnet behind the same security group. One assumes the role orders-client, the other stranger.
  2. Each signs its request with SigV4 for the service vpc-lattice-svcs and calls the service's Lattice DNS name.
  3. The request reaches the service network, whose auth policy allows any non-anonymous caller from that VPC. Both callers pass this layer.
  4. The request then reaches the rates service, whose auth policy names the orders-client role and the GET method.
  5. orders-client gets 200; stranger gets 403 β€” decided by AWS, before either request reaches any code.
  6. Allowed requests pass through a listener on port 80 to a Lambda target group and the backend function, which performs no authorisation of its own.

Design decisions

A Lambda target group, not an ALB. Lattice can forward to IPs, instances, an ALB or a Lambda function. Choosing Lambda means the week runs with no load balancer and no servers β€” you pay for neither idle compute nor an idle ALB.

Two layers of policy, not one. The service network expresses "everything in this organisation"; the service expresses "this role, this method". Separating them is what makes the failure in Troubleshooting both possible and instructive.

No authorisation in the application. Deliberate β€” the point is that the check happens in AWS, so the backend cannot forget to implement it.

Prerequisites

RequirementDetail
Terraform>= 1.10
AWS provider>= 6.68 β€” auth_type is optional on both Lattice resources, which matters below
HCP TerraformA workspace with OIDC to AWS. No static keys
AWS permissionsCreate VPC Lattice, Lambda, VPC, IAM and CloudWatch Logs resources
Regionus-east-1

One region assumption will bite you. If your account has opted into a Local Zone, data.aws_availability_zones returns it alongside the standard zones β€” and Lambda cannot attach an ENI in a Local Zone. Filter on zone type rather than taking the list as given.

No prerequisite labs. Week 23 is self-contained.

Implementation

Twenty-four resources: a VPC with two subnets, a service network and its VPC association, a Lattice service with listener and Lambda target group, two auth policies, three Lambda functions, and the IAM roles that separate them. The lab guide carries the full Terraform and exact commands.

week-23-service-networking-lattice/
|
+-- terraform/
|   +-- environments/dev/   HCP backend, OIDC, wires the modules
|   +-- modules/network/    VPC and subnets, zone-type filtered
|   +-- modules/lattice/    service network, service, listener, both auth policies
|   +-- modules/compute/    the three Lambda functions and their roles
|
+-- src/                    byte-identical caller code, two different roles
|
+-- scripts/                the six signed test calls
Only the directories that matter. The full repository is on GitHub.

The two callers are 2 of the 24 resources and differ in exactly one attribute β€” their execution role. Same code, same subnet, same security group. That is what makes the test meaningful: anything that distinguishes them has to be identity.

The control itself is two arguments and a document:

auth_type = "AWS_IAM"      # on the service AND the service network
# plus an aws_vpclattice_auth_policy naming the role and method

Both are optional in the provider, and the auth policy is a separate resource that does not set auth_type itself. Omit it and you get the default, NONE.

The applied HCP Terraform run for week-23-dev
Applied. Two apply failures got us here β€” see Challenges.

Validation and Evidence

Deployment succeeding proves nothing about authorisation. Six signed calls were made against the running service.

TestExpectedActualStatus
Infrastructure deploys24 resources24 created, apply cleanPass
orders-client calls GET200200Pass
stranger calls GET403403, "no service-based policy allows"Pass
Unsigned (anonymous) call403403, "no network-based policy allows"Pass
Policy deleted, auth_type still AWS_IAMFails closed403 β€” fails closed as documentedPass
auth_type set to NONE, policy left attachedPolicy still enforced200 for the wrong roleFail β€” and that is the finding
The VPC Lattice console showing the rates service Access tab: auth type AWS IAM, the auth policy reporting Active, and the policy JSON naming the orders-client role and a condition restricting the request method to GET
Layer two, in the console. Auth type AWS IAM, the policy Active, and inside it one role and one method. The service network carries a second policy of its own, and each object has its own authType deciding whether that policy is read at all.
Terminal output showing the same request from two Lambda functions: 200 from the orders-client role and 403 from the stranger role, plus a 403 for the wrong method and a 403 for an unsigned call
A security group cannot produce these four answers. It sees one subnet, one set of rules, and four identical packets.
The six authorisation tests with pass and fail markers, and the three connectivity and signing faults found by running them
Every one of these was found by calling the service, not by reading the configuration.

Security and Cost

Security

Identity, not address. The control this lab exists to demonstrate is that "only role X may call method Y" is expressible and enforced by AWS before the request reaches any code. A security group cannot express it.

Two layers, evaluated independently. Both must allow a request. The denial messages name which layer refused β€” "no network-based policy allows" versus "no service-based policy allows" β€” which is a genuinely useful diagnostic.

No static credentials. HCP Terraform assumes its AWS role by OIDC.

The accepted trade-off: an auth policy controls who may call, not what the call contains. It is not a WAF and does not inspect payloads.

Not production-ready as built. A real deployment needs the auth policy under the same review as any other IAM policy, and auth_type asserted explicitly on every Lattice resource rather than left to the provider default β€” see Troubleshooting for why that is not a style preference.

Cost

$1.00 measured, verified against Cost Explorer by usage type: 21 service-hours on 15 October and 19 on the 16th, at $0.025 each, under USE1-VPCLattice-Service-Hourly. Data processing came to $0.0000.

Cost Explorer output grouped by usage type showing VPC Lattice service-hourly charges of 21 hours on 15 October and 19 hours on 16 October totalling one dollar, with 35 requests billed free and data processing at one millionth of a dollar
Read from the bill, grouped by usage type. Four lines, one of which is 99.9999% of the total: the service existing.

That shape is the thing worth taking away. A Lattice service bills for being reachable, not for being called β€” an idle service network costs exactly what a busy one does. The only variable that moves the number is how long it is left standing, which is why the week ends with a destroy run rather than a quiet weekend.

A resource gateway is billed at $0.10 per resource-hour, four times the hourly rate of a service, which is why this build does not use one.

Troubleshooting and Lessons

A policy that is attached, visible, and inert

Problem. With auth_type set to NONE, the wrong role using the wrong method received a 200.
Root cause. auth_type and the auth policy are separate objects. The policy stayed attached and visible, reporting state: Inactive β€” a field almost nobody queries β€” while the service reported Status: Active at the top of the same page.
Resolution. Set auth_type = "AWS_IAM" explicitly on both the service and the service network. Terraform makes it optional on both, and the auth policy resource does not set it, so omitting it silently yields the NONE default.
Lesson. An attached policy is not an enforced one. Query the state field, not the document.

The same console Access tab as earlier, now with auth type None: the identical policy JSON is still attached and still listed, but greyed out and labelled Inactive, while the service itself still reports status Active
The same screen as before. The same policy. Two words different. Auth type reads None, the policy reads Inactive, and the console greys the document out rather than removing it. The service still reports Status: Active at the top of the page.

And the system still looked healthy

Problem. With the service policy inert, spot-checking from outside the VPC still returned 403, suggesting the controls were fine.
Root cause. The service network policy carried on refusing anonymous callers even though the service layer was disabled. Something was still saying no.
Resolution. Test from inside the VPC with a real signed identity, not from outside with none.
Lesson. A half-disabled two-layer control looks healthier than a fully disabled one, because part of it still refuses. That is worse than an obvious failure, not better.

Three that cost an evening

An opted-in Local Zone. data.aws_availability_zones returned us-east-1-dfw-1a, a subnet landed there, and Lambda refused to attach an ENI. Filtering on zone type fixes it portably.

The VPC association security group needs inbound rules. The client makes an outbound call, so an egress-only group looks obviously right. It produces no error at all β€” requests simply time out, which sends you hunting for a routing problem.

Lattice requires x-amz-content-sha256: UNSIGNED-PAYLOAD. Omit it and you get IncompleteSignatureException; supply a real hash and you get "Signed payloads are not supported". Both arrive as 403s from a service whose entire job is issuing 403s β€” a bad costume for a signing bug to wear.

Every one of these was found by calling the service, not by reading the configuration.

Cleanup and Key Takeaways

A VPC cannot be deleted while a service network association survives, so a half-finished destroy leaves a VPC that looks orphaned and is not. The teardown was run through HCP β€” run-F6yePbXRy6hMQK3m, 32 resources β€” so it has an id someone can look up later.

"Applied" is Terraform's account of itself. The verification puts the same questions to AWS: service networks, services, both association types, target groups, the VPC, the Lambdas, the log groups and the IAM roles.

Nine AWS CLI commands run after the destroy, each counting a different resource type that the week created, and each returning zero, including a tag-based search across all services for Week equals 23
β€œApplied” is Terraform's account of itself. These are the same questions put to AWS, including one that does not name the week's resources at all β€” a tag search, in case something was created outside the naming convention.

Nothing is shared and nothing survives. Left running, the only meter is the service itself at $0.025 per hour.

Key takeaways

  • An auth policy lets you write the sentence you mean. "Role orders-client may GET this service" is not expressible in a security group, and it is enforced before the request reaches your code.
  • auth_type and the policy are different objects that can disagree. A policy can be attached, visible and inert. Assert auth_type explicitly β€” the provider default is NONE.
  • A half-disabled layered control looks healthier than a broken one. Part of it still refuses, which is exactly what makes it dangerous.
  • Lattice bills for reachability, not traffic. An idle service costs what a busy one does; only elapsed hours move the number.
  • Signing failures wear the same costume as authorisation failures. Both are 403. Read the message, not the status code.

References

Comments

How was your experience?
Your feedback helps improve this site.
PoorExcellent