AWS Daily Intelligence #55 - Advisory, and on the critical path

Resilience Hub now attaches an AI investigation to every resilience test run on a connected service. The documentation is unusually careful about what that investigation is worth, and in the same breath puts it between you and your test report.

“The DevOps Agent verdict is advisory; your test run's own result remains authoritative.”

Executive summary

“The next generation of AWS Resilience Hub now integrates with AWS DevOps agent allowing you to review failure mode assessment findings and implement mitigations to resolve findings and review resilience test results.”

Two surfaces. On a finding, you press a button: “Chat with DevOps Agent”, which is “available on every failure mode assessment finding”. On a test, you press nothing — “a DevOps Agent investigation starts automatically with the test run, and a summary of its analysis becomes available shortly after the test completes”.

The sentence worth keeping is the boundary AWS draws around the output. The agent returns a “verdict, confidence, and analysis summary” — the first two words borrowed from a system that decides things — and the docs then say plainly that it decides nothing.

What changed

Aspect	What AWS says
On a test run	“Resilience Hub now launches an AWS DevOps Agent investigation that observes the test and returns a structured analysis”
Trigger	Automatic: “starts automatically with the test run”
What it produces	“symptoms, root cause, and impact along with observed recovery actions, like ARC Region switch”
Standing of that output	“The DevOps Agent verdict is advisory; your test run's own result remains authoritative”
On a finding	“Chat with DevOps Agent… available on every failure mode assessment finding”, with “the finding's context already in place”
Reports	“test report generation completes when DevOps Agent completes the investigation”
Identity used	“assuming the same invoker role that you have already set up at service level”
Enablement	Per service, via “the Agent Space ARN in the service's DevOps Agent configuration”
Availability	15 Regions for Resilience Hub; the agent “can investigate resources in any Region regardless of where the Agent Space is created”

Architecture

Two different integration shapes, and the difference matters more than the feature list suggests:

On a finding, it is pull. You open a finding, you choose to chat, a session opens with context loaded. Nothing happens if you do not ask.
On a test, it is push. Connecting the service once means every subsequent test run carries an investigation. There is no per-run flag in the documented flow.

And then the dependency. The agent cannot change the test's answer — but “test report generation completes when DevOps Agent completes the investigation”. The component that is explicitly not authoritative is nonetheless between you and the authoritative document.

Business value

A resilience test produces a result. What it has not produced is an explanation, and the gap between those two is where most of the work has always lived — somebody reads dashboards for an hour afterwards and writes down what they think happened.

That is precisely what is being automated: “symptoms, root cause, and impact along with observed recovery actions, like ARC Region switch”. An agent that watched the test while it ran, with access to the telemetry, is a reasonable thing to ask for a first draft of that narrative.

The finding-side integration is the quieter win. A failure mode assessment produces findings that are accurate and abstract, and the distance from "this dependency is single-AZ" to "here is what we change on Thursday" is the part teams stall on. Opening a session “with the finding's context already in place” removes the step where somebody re-explains the finding to the thing that is going to help with it.

Security considerations

This is the part the documentation gets right, and it is worth reading as a template rather than skimming.

“Resilience Hub connects to your Agent Space by assuming your existing invoker role — the same cross-account role that resource discovery uses. You do not create a new role or change its trust policy.”

Only permissions are added, and they are enumerated: CreateBacklogTask, GetBacklogTask, ListExecutions, ListJournalRecords and UpdateBacklogTask for the investigation itself, plus GetAgentSpace, GetOperatorApp and ListAssociations for validation. Five and three — and AWS says so: “These eight actions are the complete set that Resilience Hub calls on your behalf… None of them can mutate your Agent Space, or your resources.”

Then the sentence that makes this unusual:

“Do not attach the broad AWS managed policies AIDevOpsAgentFullAccess or AIDevOpsOperatorAppAccessPolicy. They grant mutating Agent Space actions that Resilience Hub must never call.”

A service team naming the AWS managed policies that will work and telling you not to use them is rare. The easy path for a feature like this is a new managed policy with everything in it and a note that you may scope it down. This does the opposite, and it is the right instinct: an agent that reads your telemetry should not hold a grant that lets anything reconfigure the agent.

The residual question is one of blast radius rather than permissions. The role doing this is the cross-account discovery role, which already reaches across your estate. Adding read-and-poll on agent tasks to it is small; it is still a widening of what that one identity can do, and the trust policy did not change so nothing in your change review will flag it.

Cost considerations

The announcement states no price for the integration itself and the documentation does not either, so treat the cost as whatever DevOps Agent charges for an investigation, multiplied by a number you no longer choose per run.

That multiplication is the only cost point worth making. Because the investigation “starts automatically with the test run”, the volume is a function of how often you test, and teams that run resilience tests on a schedule across many services will generate investigations on the same schedule without an intervening decision. Check the Agent Space pricing before connecting a service whose tests run nightly.

Operational considerations

The dependency is the operational story. “Test report generation completes when DevOps Agent completes the investigation.”

If a test report is an artefact somebody collects — a quarterly DR evidence pack, a change-approval attachment, a compliance upload — then its availability is now a function of an investigation finishing. Nothing in the documentation suggests that is slow, and the phrasing elsewhere is “shortly after the test completes”. But a pipeline that polls for the report is now polling for two things.

The failure mode is named, which is helpful: “If the invoker role is missing these permissions, the connection fails with an access denied error, and the service's DevOps Agent status becomes FAILED with a reason that tells you to update the role. This is the most common cause of a failed connection.”

And the exit is clean: “To disconnect a service later, update it with an empty DevOps Agent configuration”, which “clears the connection without affecting the Agent Space itself”. Per-service connection and per-service disconnection means you can pilot this on one service without an account-wide decision.

Tradeoffs

What you gain	What it costs
A written analysis of every test run, without anyone writing it	An investigation per run, whether or not the run was interesting
Verdict and confidence surfaced beside the result	Two verdicts on one page, and a reader has to know which one counts
Reports that carry the narrative, not just the outcome	Report generation waits on the investigation
No new role, no trust policy change	The discovery role's permissions widen, invisibly to change review
Interactive investigation seeded from a finding	A plausible answer is harder to disbelieve than a blank page

The second row is the one to think about. A console page showing a test result and, beside it, a verdict and a confidence score is a page where a reader can take away the wrong number — particularly a reader who was not in the room when the separation was explained. The documentation is clear; the screen is where it has to survive.

Implementation guidance

Create an Agent Space and note its ARN, of the form arn:aws:aidevops:region:account-id:agentspace/agent-space-id.
Add the eight actions to the existing invoker role as an inline policy. Not the managed policies — AWS names them specifically so you do not reach for them.
Connect one service with the Agent Space ARN in its DevOps Agent configuration, and confirm with GetService that the status is not FAILED.
Run a test you already understand. The value of the first few investigations is calibration — you are reading the agent's analysis of something you can already explain.
Check what consumes your test reports before connecting a service whose reports feed a schedule.
Try the finding side separately. Chat is available on every finding and changes nothing about your tests.

Best practices

Keep the verdict in its place. AWS put the sentence in the documentation; put it in your runbook too, where the person reading the console will see it.
Grant the eight actions, not the managed policies. The inline policy is published in full; copy it.
Note the role widening in your own records even though no trust policy changed. Nothing else will.
Pilot per service. The configuration is per service in both directions, which makes a narrow trial genuinely narrow.
Read the analysis against a test whose outcome you know, before reading one against a test whose outcome you do not.

Who should adopt this and when

Now, if you already run next-generation Resilience Hub with discovery configured. The prerequisite is an invoker role that exists, and the change is an inline policy plus one field on a service. That is a small enough step to take on one service this week.

Soon, if resilience testing is something you do and then interpret by hand. The gap this fills is real, and the honest framing of the verdict means you are not being asked to trust it further than it deserves.

Later, if your test reports feed a compliance process with timing expectations. Understand the new dependency before you introduce it into a chain somebody audits.

Not yet, if you are not on the next generation of Resilience Hub. The integration is described throughout as a next-generation feature, and the migration is the larger decision.

Key takeaways

Resilience Hub now launches a DevOps Agent investigation that “observes the test and returns a structured analysis”, automatically, once a service is connected.
“The DevOps Agent verdict is advisory; your test run's own result remains authoritative” — the right separation, stated outright.
But “test report generation completes when DevOps Agent completes the investigation”, so the advisory component gates the authoritative artefact.
It runs on your “existing invoker role” with no new role and no trust policy change — eight added actions, none of which “can mutate your Agent Space, or your resources”.
AWS names two managed policies and tells you not to attach them, which is the opposite of the usual default and worth copying as a pattern.
Connection and disconnection are both per service, so a pilot is cheap and reversible.