Home› Blog› AWS Daily Intelligence #80 - A fifth status check,…
AWS Daily Intelligence AWS

AWS Daily Intelligence #80 - A fifth status check, and it increments the metric your automation already watches

Verified against current vendor documentation on 20 November 2026. Pricing, limits and API behaviour were checked against the official docs on that date. Cloud services change fast — if you are reading this much later, treat the specifics as a starting point and re-check the linked sources.

EC2 gained a performance status check for accelerated instances: hardware degradation that leaves the instance running and reachable. The capability is genuinely new. The thing to check today is what it does to an alarm you wrote years ago.

“If a performance status check fails, Amazon EC2 increments the StatusCheckFailed_Performance metric. A failed performance status check also increments the StatusCheckFailed metric.”

Executive summary

“EC2 performance status checks identify nonterminal hardware conditions that impact performance on accelerated computing instances”, and “These checks also recommend an action, such as rebooting to complete GPU memory row remapping without replacing the instance.” They are “available at no additional charge”, in “all commercial AWS Regions and the AWS GovCloud (US) Regions”.

Alongside them, Detailed Monitoring gained “support for OpenTelemetry metrics, one-minute GPU metrics that help teams tune accelerated workloads, and a new, higher resolution 10-second CPU utilization metric that helps customers size and scale compute capacity using EC2 Auto Scaling”, which are “opt-in and incur standard CloudWatch charges”.

The documentation then supplies two facts the announcement does not. EC2 now has “five types of status checks: system, instance, attached EBS, application, and performance” — most people carry two in their heads. And the new check feeds the aggregate metric that existing automation is built on.

What changed

Every status check before this one answered a binary question about availability. System checks cover “the AWS systems on which your instance runs” and “detect underlying problems with your instance that require AWS involvement to repair”. Instance checks cover “the software and network connectivity of your individual instance” and “detect problems that require your involvement to repair”. Either way, something was unreachable.

The performance check is a different category: “Performance status checks monitor for hardware issues that impact accelerated computing instance performance without making the instance unreachable. These issues do not require scheduled maintenance.”

That is a condition the model had no way to express before. A GPU quietly losing memory capacity is not a failed instance and not a scheduled retirement — it is an instance that passes every check while doing less work than you are paying for. Previously the only signal was your own benchmark numbers drifting.

And the remedy is unusually specific for a status check: “a graphics processing unit (GPU) memory error can require an instance reboot to reset the GPU and trigger row remapping, which resolves the issue without replacing the instance.”

Architecture

Diagram: Amazon EC2 now provides five types of status checks: system, instance, attached EBS, application, and performance. Status checks are performed every minute, returning a pass or a fail status, and if one or more checks fail the overall status of the instance is impaired. System, instance, attached EBS and performance checks are managed by Amazon EC2 and cannot be disabled or deleted, while application status checks are opt-in and monitor HTTP or HTTPS responses and can be suppressed during deployments. System, instance and attached EBS checks run on every instance, the attached EBS metric is available only for Nitro instances, and performance status checks run only on supported accelerated instance types. The new performance status check monitors for hardware issues that impact accelerated computing instance performance without making the instance unreachable, and these issues do not require scheduled maintenance, which is a condition the previous model could not express: an instance that passes every check while doing less work than you are paying for. The documented example is a GPU memory error that can require an instance reboot to reset the GPU and trigger row remapping, resolving the issue without replacing the instance. The PerformanceStatus object returned by DescribeInstanceStatus includes Status, ImpairedSince and Remediation fields, and when Status is impaired the Remediation field carries the recommended action, such as reboot. The critical interaction is that a failed performance status check increments the StatusCheckFailed_Performance metric and also increments the aggregate StatusCheckFailed metric, so any existing alarm watching the aggregate, including automatic instance recovery and Auto Scaling health replacement, now fires on a nonterminal condition whose documented remediation is a reboot rather than a replacement. Alongside the check, EC2 Detailed Monitoring adds OpenTelemetry metric support, one-minute GPU metrics, and a 10-second CPU utilization metric intended for sizing and scaling with EC2 Auto Scaling, all opt-in through categories applied across a fleet and incurring standard CloudWatch Detailed Monitoring charges, while the performance status checks themselves are free and available in all commercial Regions and GovCloud US.

Five checks, and what each one means

CheckCoversRuns onWhose problem
System “the AWS systems on which your instance runs” Every instance AWS — “require AWS involvement to repair”
Instance “the software and network connectivity of your individual instance” Every instance Yours
Attached EBS Whether attached volumes can complete I/O Nitro instances only Either — wait or replace
Application HTTP and HTTPS responses, opt-in Instances you associate Yours
Performance Degradation without unreachability Supported accelerated types Yours, usually a reboot

Four of the five “are managed by Amazon EC2 and cannot be disabled or deleted”, which matters for the next section: you cannot opt out of the new check, only out of acting on it.

The interaction to check today

All status checks share one aggregation rule: “Status checks are performed every minute, returning a pass or a fail status. If all checks pass, the overall status of the instance is OK. If one or more checks fail, the overall status is impaired.”

And the new check joins that aggregate. A performance failure increments its own StatusCheckFailed_Performance metric and “also increments the StatusCheckFailed metric”.

Consider what is already watching StatusCheckFailed in a mature estate. Auto-recovery alarms, since the documentation points at exactly that: “You can also create an Amazon CloudWatch alarm that monitors an Amazon EC2 instance and automatically recovers the instance if it becomes impaired due to an underlying issue.” Auto Scaling health replacement. Dashboards. Runbook triggers. None of them were written with performance degradation in mind, because it did not exist as a signal.

So on a GPU fleet, an alarm whose intent was this instance is broken, replace it can now fire for a condition whose documented fix is a reboot — one that “resolves the issue without replacing the instance”. Replacing it instead is not catastrophic, but on accelerated capacity it trades a sixty-second reboot for a scarce-instance relaunch, and it discards the specific remedy AWS went to the trouble of recommending.

The field that carries the recommendation

“The PerformanceStatus object returned by DescribeInstanceStatus includes the Status, ImpairedSince, and Remediation fields. If Status is impaired, check Remediation for the recommended action”, and “if Remediation is reboot, reboot the instance to clear the issue and return it to service without replacement.”

Two things follow. Remediation is a machine-readable instruction, so the correct automation reads it rather than guessing — this is the rare case where AWS tells you what to do in a field. And ImpairedSince gives you duration, which is what distinguishes a transient blip from a degraded instance that has been quietly underperforming for a week.

Neither field is available through the metric. The metric tells you a check failed; only DescribeInstanceStatus tells you which check and what to do about it.

The metrics shipped alongside

“support for OpenTelemetry metrics, one-minute GPU metrics… and a new, higher resolution 10-second CPU utilization metric”, with the purpose stated: the 10-second metric “helps customers size and scale compute capacity using EC2 Auto Scaling”.

A 10-second CPU metric is a meaningful change for scaling policy, because it matches the resolution a high-resolution CloudWatch alarm can evaluate at. The one-minute default has always been the floor on how fast a reactive scaling decision could be made from CPU; this lowers it. The opt-in shape is fleet-friendly — “Customers can selectively opt into these new metrics using categories that they can easily apply to their instances across their fleet” — and the cost is the usual Detailed Monitoring bill rather than something new.

Business value

The value is concentrated and large for anyone running accelerated capacity. A GPU with remappable memory errors degrades throughput without failing anything, so the cost shows up as training jobs taking longer or inference latency creeping — attributed to the model, the data or the code long before anyone suspects the hardware. A check that names it, with a reboot that fixes it, converts an unexplained performance regression into a one-line operation.

It is also free and cannot be turned off, so the signal exists on every supported instance whether or not anyone acts on it. That makes it available retrospectively: the next time a GPU job is inexplicably slow, ImpairedSince is a question you can now ask.

The 10-second CPU metric is narrower but real for latency-sensitive autoscaling, where a minute of averaging is the difference between scaling during a spike and scaling after it.

Security considerations

Little changes here, and that is worth stating. These are platform-side observations delivered “without setup within the instance”, so no agent, no in-guest credential and no new network path. For a hardened or regulated fleet that is the appealing property: visibility that does not require installing anything.

Reading the recommendation needs an API permission, not an agent. The Remediation field comes from DescribeInstanceStatus, so whatever automation acts on it needs that call rather than host access.

Application status checks are the one with a new exposure, since they probe an HTTP path you nominate. Worth noting they can be quieted deliberately: “You can suppress application status checks during application restarts or deployments to avoid false-positive failures during expected downtime.”

Cost considerations

The split is clean. Performance status checks are “available at no additional charge”. The metrics are not: “EC2 Detailed Monitoring metrics are opt-in and incur standard CloudWatch charges for EC2 Detailed Monitoring.”

The cost to model is the 10-second CPU metric across a large fleet. Higher resolution means more datapoints per instance per hour, and the saving it enables — scaling sooner, so running less idle capacity — has to cover that. On a small fleet of latency-sensitive instances the trade is obvious; applied fleet-wide by default it is a bill nobody asked for, which is presumably why the opt-in is category-based rather than global.

Against that, the performance check can save real money on accelerated instances, where a reboot instead of a replacement avoids both the relaunch and the risk of not getting the capacity back.

Operational considerations

Audit every alarm on StatusCheckFailed first. On accelerated instances that metric now has a new contributor with a different correct response. This is the one action to take this week.

Split your alarms by metric rather than using the aggregate. StatusCheckFailed_System, StatusCheckFailed_Instance, StatusCheckFailed_AttachedEBS and StatusCheckFailed_Performance each imply a different action, and the aggregate deliberately erases that distinction.

Have automation read Remediation instead of assuming. It is a field designed to be read, and the documented value reboot is not what most impairment automation does.

Remember the check cannot be disabled. Four of the five are “managed by Amazon EC2 and cannot be disabled or deleted”, so the signal arrives whether your runbooks are ready or not.

Treat ImpairedSince as the triage field. A minute of impairment and a week of it are different incidents with the same metric value.

Tradeoffs

ChoiceWhat you gainWhat it costs
Alarm on aggregate StatusCheckFailed One alarm covers every impairment type Performance degradation triggers replace-the-instance logic
Alarm per specific metric Each response matches the actual fault Four alarms per instance instead of one
Automate on Remediation Does what AWS recommends, including reboot-not-replace Needs DescribeInstanceStatus in the loop, not just a metric alarm
10-second CPU metrics fleet-wide Faster, more accurate scaling decisions Six times the datapoints of one-minute monitoring, billed
10-second CPU on latency-sensitive groups only The benefit where it pays Category management per fleet segment

Implementation guidance

Inventory your StatusCheckFailed alarms and note which touch accelerated instances. Those are the ones whose meaning changed without anyone editing them.

Add a StatusCheckFailed_Performance alarm whose action is a reboot, and exclude performance from whatever currently replaces instances, so the two paths do not race.

Query DescribeInstanceStatus for PerformanceStatus on your GPU fleet now. The check is already running and free, so any existing impairment is visible today — and ImpairedSince may explain a performance complaint you have been carrying.

Opt into the 10-second CPU metric by category, for the groups whose scaling is actually latency-bound, rather than everywhere.

Best practices

Keep the per-check metrics as your automation inputs and the aggregate as a dashboard summary. The aggregate is a good human signal and a poor machine one, precisely because it unions conditions with incompatible remedies.

Write the reboot path down before you need it. A reboot on an accelerated instance to trigger row remapping is a legitimate, documented repair, and it will not be anyone's instinct at three in the morning.

When a GPU workload slows down, check instance status before re-profiling the model. That ordering is new, cheap, and the whole point of the feature.

Who should adopt this and when

Act now if you run accelerated instances at all. Not to enable anything — the check is already on and free — but to check what your existing aggregate alarms will do with it.

Act now if any automation replaces instances on StatusCheckFailed. That is the specific combination where this launch changes behaviour you did not choose.

Adopt the 10-second CPU metric selectively, where scaling latency is a known problem rather than as a default.

Nothing to do on non-accelerated fleets, where performance status checks do not run — though the five-check model is still worth knowing, since most mental models stop at two.

Key takeaways

The new check describes a state the old model could not: “hardware issues that impact accelerated computing instance performance without making the instance unreachable”, with a Remediation field that may say reboot and resolve it “without replacing the instance”.

The line that changes existing behaviour: “A failed performance status check also increments the StatusCheckFailed metric.” Alarms written for terminal failures now fire for a degraded-but-working instance.

And EC2 has “five types of status checks: system, instance, attached EBS, application, and performance” — four of them mandatory, one opt-in, and only one of them about speed.

Official AWS references

Comments

How was your experience?
Your feedback helps improve this site.
PoorExcellent