Policy met means estimated to meet

Business Challenge

#107 ended on the primitives that survived two withdrawn application abstractions: a tag key, a resource group, a search. Resilience Hub is the first thing in this run that consumes one of those and produces a verdict — it takes an application definition and tells you whether the application meets its recovery targets.

The verdict is worth reading literally. AWS defines the passing status as: “The application is estimated to meet its RTO and RPO targets defined in the policy.” And the failing one symmetrically: “The application could not be estimated to meet the RTO and RPO targets defined in the policy.”

Both statuses describe what an estimator concluded from configuration. Neither describes a recovery. That distinction is the whole post, and unusually, the service is the one making it.

1The green status contains its own caveat, and hands the work back

Read the full definition of Policy met, including the example AWS chose:

“The application is estimated to meet its RTO and RPO targets defined in the policy. All its components meet the defined policy objectives. For example, you selected an RTO and RPO target of 24 hours for disruptions across AWS Regions. AWS Resilience Hub can see that your backups are copied to your fallback Region. You are still expected to maintain a recover from a backup standard operating procedure (SOP), and to test and time it.”

That last sentence is inside the definition of the passing state. The service can see that backups are being copied; it cannot see whether anyone can restore from them inside 24 hours, and it says so rather than implying otherwise. What Policy met asserts is that your configuration is consistent with your target. What it does not assert is that the recovery works, or that anyone has ever performed it.

Even the testing path keeps the word. “Tests for SOPs measure estimated workload RTO and estimated workload RPO” — so a FIS experiment against an SOP produces a measured number that AWS still labels an estimate of the workload's objective, because one timed run of one procedure under one injected fault is evidence, not a guarantee.

Fix

Treat Policy met as a configuration review that passed, and record the date of the last timed restore separately. If those two facts live in the same report, the status stops being read as a recovery capability. The gap AWS names in the definition — test it and time it — is what converts an estimate into a number you can defend in an incident review.

2Region targets are optional, so the green status may be silent on Regions

A policy carries RTO and RPO per disruption type, and the types are not equally mandatory. The console walks you through application, infrastructure and Availability Zone values, and then:

“(Optional) If you have a multi-Region application and if you want to define a Region RTO and RPO, turn on Region - Optional.”

So the scope of Policy met is whatever disruption types your policy defines. An application with no Region targets can report Policy met while nothing has been estimated about a Region incident at all — and the status is a single value, so the omission is not visible in it.

This compounds with the defaults. “We have pre-populated a default RTO and RPO for your application”, which means a policy created quickly carries numbers nobody chose. A green status against a pre-populated target over three of four disruption types is a sentence with a lot of hidden qualifiers.

Fix

Write the policy's coverage next to its status wherever it is reported: which disruption types carry targets, what the values are, and whether anyone chose them. Policy met is not a fact about an application; Policy met for application, infrastructure and AZ disruptions at a 24-hour RTO, with no Region target defined is one.

3The estimate is bounded by what you imported, and the sources are not equivalent

The estimate is explicitly a function of the definition: “the RTO that your application is estimated to meet based on the imported application definition and then run an assessment”. There are four resource collection sources, and they differ in ways that outlive the import.

Resource Groups is #106's object — a stored query whose membership is computed when something reads it. CloudFormation stacks change through deployments. Terraform state files are a snapshot in S3, so the definition is as current as the last state file you pointed at. And myApplications is the product #107 covered, which “no longer allows creation of new applications or updates of existing applications” — so that source can still be imported from, and can no longer change.

EKS is different again, and manual: “For AWS Resilience Hub to assess your Amazon EKS clusters, you must manually add the relevant namespaces to each of the Amazon EKS clusters”. A namespace you did not add is a namespace outside the assessment, and nothing in the compliance status distinguishes that from a namespace that passed.

Fix

Record which source an application was defined from, because it determines what keeps the definition true. A stack-based definition drifts when someone deploys; a Resource Groups definition drifts when someone edits a tag; a Terraform definition drifts the moment the state file moves on. Drift detection catches resource additions and deletions, but it cannot tell you the question you asked was too narrow.

Architecture

Diagram: AWS Resilience Hub reports a compliance status against the recovery targets in a resiliency policy, and both the passing and failing statuses are outputs of an estimator reading configuration rather than records of a recovery. Policy met is defined as the application being estimated to meet its RTO and RPO targets, and that same definition adds that you are still expected to maintain a recover from a backup standard operating procedure and to test and time it. Policy breached is defined symmetrically as the application not being able to be estimated to meet the targets. Two further statuses exist: Not assessed, and Changes detected, meaning there is a new published version of the application that has not yet been assessed. The estimate is bounded three ways. First by the policy's coverage, because RTO and RPO are set per disruption type across Application, Cloud Infrastructure, Cloud Infrastructure AZ disruption and Cloud Infrastructure Region incident, and the Region values are optional, so a green status can be silent on Region incidents entirely while remaining a single value. Second by the estimator's own floor: a target of zero is accepted but the lowest possible assessment result is near zero, so a zero target always produces Policy breached. Third by the imported application definition, which can come from AWS Resource Groups, CloudFormation stacks, Terraform state files limited to 4 megabytes, or myApplications, a product that no longer allows creation of new applications or updates of existing ones, while EKS clusters require namespaces to be added manually. Drift detection compares against the previous assessment in which the component met the policy, not the most recent assessment. The resiliency score measures how closely the application follows AWS recommendations for the policy, alarms, standard operating procedures and tests, rather than measuring resilience. Quotas include 50 applications and 10 resiliency policies per Region, 400 resources and 200 Application Components per application, 200 assessments per application per month, a non-adjustable limit of one concurrent assessment per application, and a non-adjustable 365-day retention of past assessments.

Four statuses, and what each one actually asserts

Status	AWS's definition	What it asserts
Policy met	“estimated to meet its RTO and RPO targets”	The configuration is consistent with the targets that were defined
Policy breached	“could not be estimated to meet the RTO and RPO targets”	The estimator could not get there — which includes the zero-target case below
Not assessed	“It's not currently assessed or tracked”	Nothing. Usefully explicit, unlike an absent result
Changes detected	“There is a new published version of the application that has not yet been assessed”	The last verdict describes an earlier version of the application

Changes detected is the one worth wiring an alert to. It is the state where a status is still displayed and is no longer about the thing in front of you.

A target of zero is accepted and always fails

“AWS Resilience Hub allows you to enter a value zero in the RTO and RPO fields of your resiliency policy. But, while assessing your application, the lowest possible assessment result is near zero. Hence, if you enter a value zero in the RTO and RPO fields, the estimated workload RTO and estimated workload RPO results will be near zero and the Compliance status for your application will be set to Policy breached.”

This is the clearest available demonstration that the status is an estimator's output rather than a measurement. A zero RTO is an aggressive engineering goal; here it is an arithmetically unreachable one, because the estimator's floor is near zero and near zero exceeds zero. The breach reports a property of the comparison, not of the application.

Worth knowing before someone sets an aspirational zero in a policy and reads the resulting red status as a finding about the architecture.

The four disruption types, and which is optional

Disruption type	AWS's description	Target required?
Application	“The infrastructure is healthy, but the application or software stack doesn't operate as needed”	Yes
Cloud Infrastructure	“resolved by rebooting, recycling, or reloading the faulty components”	Yes
AZ disruption	“resolved by switching to a different Availability Zone”	Yes
Region incident	“resolved by switching to a different AWS Region”	Optional

Splitting the question four ways is the right design — an RTO for a bad deployment and an RTO for a Region incident are different engineering problems and should not share a number. The cost is that the compliance status collapses them back into one value, and the one that may be absent is the most expensive to get wrong.

Drift is measured against the last passing assessment

“For comparison, AWS Resilience Hub uses the previous assessment in which the application component met the policy.”

Not the previous assessment — the previous passing one. For a component that has been breaching for two months, policy drift is computed against a green state from before that, which is the right baseline for the question what changed when this broke and the wrong one for what changed since yesterday. Both are reasonable questions; only one is answered.

The resiliency score is not a resilience measurement

“AWS Resilience Hub generates a score that indicates how closely your application follows our recommendations for meeting the application's resiliency policy, alarms, standard operating procedures (SOPs), and tests.”

The score measures recommendation adoption. That is a reasonable thing to measure and a reasonable thing to improve, and it is not the same quantity as resilience — an application can follow every recommendation and still have an untested SOP, which is the specific gap the Policy met definition names. Reading the score as a resilience percentage is the second deflation in this post, and it is the one more likely to end up on a slide.

The quotas, and the one that paces everything

“Number of applications | Each supported Region: 50” and “Number of resources per application | Each supported Region: 400”, both adjustable — a documented ceiling of 20,000 assessed resources in one Region, with “Number of Application Components per application | Each supported Region: 200” grouping them.

The one that shapes practice is not adjustable: “Number of concurrent assessments per application | Each supported Region: 1”. One assessment at a time per application, so a pipeline that assesses on every change serialises against itself. The monthly allowance is generous by comparison — “Number of assessments per application per month | Each supported Region: 200”, which is 2,400 a year against a retention period of “365” days, also not adjustable. Assessment history older than a year is gone, so a year-on-year resilience trend has to be exported before it expires.

Also fixed at ten: “Number of Resiliency Policies | Each supported Region: 10”, adjustable, which is few enough that policies are shared across applications rather than written per application — worth knowing, because a shared policy means a shared definition of what Policy met meant.

One curiosity for anyone importing large state files: the Terraform ceiling is documented twice, as “The limit for Terraform state files is 4 megabytes (MB)” in the console flow and as “4,194,305” bytes in the quota table. The second is one byte above 4 MiB. If you are near the limit, trust the byte count.

Why This Architecture Holds Up

Saying estimated, every time, is the honest choice

The word appears in the status definitions, in the metric names, and even in the output of a timed FIS test. A service that reported RTO: 24 hours as a flat fact derived from configuration would be more satisfying to read and considerably more dangerous. Resilience Hub consistently declines to claim more than it knows, and the caveat inside Policy met is the strongest version of that.

Estimating from configuration is the only thing that scales

The alternative to an estimate is a test, and a test costs a disruption. Assessing 20,000 resources by actually failing them over is not a thing anyone will do on a schedule, so a configuration-derived estimate is what makes continuous assessment possible at all. The design is right; the reading of its output is where the risk sits.

FIS integration is the bridge, and it is in the right place

Recommending experiments that “Verify that recovery procedures, or standard operating procedures (SOPs), work correctly” is the service closing its own gap — the estimate tells you where to look, and the experiment tells you whether it works. An assessment tool that stopped at the estimate would leave the reader with no path from one to the other.

Key Architecture Decisions

Decision	Choose this	Because
Reporting a compliance status	Report it with the policy's coverage and the date of the last timed restore	Policy met is an estimate from configuration, and its own definition asks you to test and time the SOP
Region RTO and RPO	Define them, or state explicitly that you have not	They are optional, and the status is one value that does not show the omission
Setting an aspirational zero target	Don't — use the smallest real number instead	The estimator's floor is near zero, so a zero target always returns Policy breached
Choosing an input source	Pick the one whose change mechanism you can see	A stack changes on deployment, a resource group on a tag edit, a Terraform state file whenever it is rewritten
Defining from myApplications	Treat it as frozen input	That product no longer accepts new applications or updates to existing ones
EKS applications	Enumerate namespaces deliberately and review the list	Namespaces are added manually, and an omitted one is simply outside the assessment
Assessing from a pipeline	Serialise, and export history before it ages out	One concurrent assessment per application, not adjustable, and 365-day retention, also not adjustable
Using the resiliency score as a target	Treat it as recommendation adoption, not resilience	That is what AWS says it measures

The audit worth running

Open your policies and check whether Region targets are set. There are at most ten of them per Region by default, so this is a short list. Any policy without Region values is one whose green status says nothing about a Region incident, and that is usually the disruption the policy was created for.

Then find the date of the last timed restore for each application reporting Policy met. If no such date exists, the status is a configuration review and should be described as one. This is the check the Policy met definition is explicitly asking for.

Finally, look for applications sitting in Changes detected. That status means a published version has not been assessed, so whatever verdict is on screen describes a previous version. Of the four, it is the one whose problem is staleness rather than scope: Policy met misleads when it is read as a recovery capability, but it is at least about the application in front of you. Changes detected is not.

Closing Thought

The thread through this run has been instruments whose output is narrower than it reads. A tag policy that cannot require a tag. An index that answers partially and completely in the same format. A group that is a question rather than a set. An application that was a tag with an ARN in it.

This one is the most carefully labelled of them and still the easiest to misread, because the label is a single word in the middle of a sentence. Estimated to meet and meets differ by everything that matters in an incident, and only one of them is what a dashboard shows. AWS put the remedy inside the definition of its own passing status — test it, and time it — which is as clear as a vendor can reasonably be about the limits of what it can tell you.

So the practical ending is small and it is not a configuration change: write the date of your last timed restore next to the status, every time the status is reported. If the two ever disagree, the restore is the one that is true.

Next in this series

Reliability — AWS Fault Injection Service: what an experiment template commits you to, how stop conditions bound the blast radius, and why an experiment that completed without tripping an alarm is two different findings depending on what the alarm was watching.

AWS Architecture Series #108 — Policy met means estimated to meet