RDS Monitoring and Performance Insights

A payments API times out for about 4 minutes every morning at 09:05. You open the RDS console. CPUUtilization peaks at 41%. FreeableMemory is flat. ReadLatency sits at 2 milliseconds. FreeStorageSpace has plenty of room. Every metric you have says the database is healthy, and the database is very clearly not healthy.

Nothing is broken in your monitoring. You are looking at the wrong layer. CloudWatch instance metrics measure the machine the database runs on. They cannot see 30 sessions queued behind a single row lock, because a lock costs no CPU, no memory, and no I/O. It costs time, and time is not something the hypervisor can measure.

RDS gives you 3 monitoring layers, each looking at the database from a different place. Knowing which one answers which question is most of the skill here.

3 layers, 3 different questions

Read that middle column again, because it is the whole lesson in one line. Layer 1 stands outside the instance and sees resource totals. Layer 2 stands inside the operating system and sees processes. Layer 3 stands inside the database engine and sees sessions. The 09:05 incident is invisible at layer 1 and obvious at layer 3.

The CloudWatch metrics worth watching

RDS sends metrics to CloudWatch in 1-minute periods by default, in the AWS/RDS namespace with a DBInstanceIdentifier dimension. Those 60-second data points stay available for 15 days.

Two of these hide traps.

DatabaseConnections is not a session count. It counts client network connections, so it excludes sessions the engine spawns for itself, job scheduler sessions, parallel execution sessions, sessions whose network connection died before cleanup, and RDS's own management connections. The real session count is higher, sometimes much higher, and sessions are what consume connection memory.

BurstBalance and EBSIOBalance% are different buckets. BurstBalance is the gp2 volume's own I/O credit bucket. EBSIOBalance% and EBSByteBalance% describe the DB instance's EBS burst capacity, which exists on many instance sizes regardless of storage type, and they are based on the throughput of every volume including the root volume. When EBSByteBalance% trends to zero, the instance is running out of computing capacity and the answer is a larger instance class, not more provisioned IOPS. Getting these 2 backwards leads you to buy the wrong thing.

Enhanced Monitoring: the view CloudWatch cannot give you

CloudWatch reads CPU from the hypervisor. Enhanced Monitoring reads it from an agent running inside the DB instance's operating system. That difference is why the 2 numbers rarely match exactly, and the gap grows on smaller instance classes where more virtual machines share one physical host.

The agent buys you something the hypervisor cannot deliver at any granularity: a per-process and per-thread breakdown. When CPU is at 90% and you need to know whether that is the database engine, a backup, or a runaway maintenance thread, this is the only layer that answers.

The details that get tested:

• Granularity is 1, 5, 10, 15, 30, or 60 seconds. Setting --monitoring-interval 0 turns it off.
• Metrics go to CloudWatch Logs, in the RDSOSMetrics log group, with a default retention of 30 days. You change that on the log group, not on the DB instance.
• It needs an IAM role. The console can create rds-monitoring-role for you; via CLI or API you create it yourself with the AmazonRDSEnhancedMonitoringRole policy and a trust relationship for the monitoring.rds.amazonaws.com service principal. The caller needs iam:PassRole.
• Turning it on does not require a reboot.
• The RDS console refreshes at best every 5 seconds. If you set 1-second granularity, you get the 1-second data from CloudWatch Logs, not from the console.

aws rds modify-db-instance \
  --db-instance-identifier payments-prod \
  --monitoring-interval 5 \
  --monitoring-role-arn arn:aws:iam::123456789012:role/rds-monitoring-role

Here is the misconception this layer creates. Because the RDS console draws Enhanced Monitoring as graphs, people assume they can alarm on those values the way they alarm on CPUUtilization. They cannot, not directly. Enhanced Monitoring output is log events, so alarming on it means creating a CloudWatch Logs metric filter over RDSOSMetrics first, then alarming on the metric that filter produces.

DB load: the metric that answers "why"

Layer 3 measures something different from every metric above. Not a resource, but work in progress.

A session is an application's conversation with the database. A session is active when it has submitted work and is waiting for an answer: either running on CPU, or waiting for a resource such as a page to be read into memory, a lock to be released, or a log write to complete. Idle sessions do not count.

DB load is the number of active sessions, and its unit is average active sessions (AAS). Performance Insights samples the count once per second. AAS is the total sessions counted divided by the number of samples.

Take 5 consecutive 1-second samples:

DB load for that interval is 2 AAS: on average, 2 sessions were active at any moment. The averaging matters. A 1-second spike to 40 sessions barely moves AAS, while 5 sessions stuck for a full minute pushes it hard. That is the right bias, because a database is hurt by sustained queueing, not by momentary bursts.

For each active session it samples, Performance Insights also captures the SQL statement, whether the session was on CPU or waiting, the host, and the user. Those 4 pieces of information are what make the number actionable rather than merely interesting.

Reading the chart: Max vCPU, waits, and top SQL

The DB load chart draws a horizontal line at the DB instance's vCPU count, labeled Max vCPU. This is the reference that turns an abstract number into a verdict.

Work through it on a db.r6g.2xlarge, which has 8 vCPUs:

• DB load steady at 3 AAS: 3 sessions active against 8 vCPUs of capacity. Comfortable.
• DB load at 8 AAS, nearly all on CPU: the instance is saturated on CPU. More vCPUs would help.
• DB load at 21 AAS with 18 of them waiting: 21 sessions are active but only a few are doing work. The other 18 are queued behind something. Adding vCPUs changes nothing, because CPU was never the constraint.

That last case is the one people misread, and it is exactly why the load is split into 2 CloudWatch metrics:

DBLoadNonCPU dominating is the signal to stop looking at instance size and start looking at what the sessions are waiting for. That is what the wait event dimension is for. Slice DB load by wait event and you usually find 2 or 3 events accounting for most of the load: I/O reads, row locks, log flushes. The specific names vary by engine, but the shape of the answer does not.

Then slice by top SQL to find which statements are producing that wait. It is common for one query out of hundreds to account for the majority of DB load. Performance Insights also captures execution plans for the most resource-intensive queries every 5 minutes, so you can see how the engine chose to run the query that is hurting you. You can also slice by host and by user, which is how you identify a single misbehaving application server or reporting account.

The workflow, in order: DB load says how bad, wait events say what kind of bad, top SQL says who is causing it, and the plan says why that query is slow.

Back to the 09:05 incident. DB load spikes to 25 AAS on an 8 vCPU instance, almost entirely on a row lock wait event, and top SQL shows a single UPDATE against the accounts table. A nightly batch job holds a long transaction, and the API's writes queue behind it. No resource metric would ever have shown you that, because no resource was under pressure.

Performance Insights is now CloudWatch Database Insights

AWS retired the Performance Insights console on 31 July 2026. The console now redirects to CloudWatch Database Insights. Nothing about the underlying measurement changed: it is the same DB load, the same average active sessions, the same wait events and top SQL. What changed is where you look at it and how it is packaged.

The things worth remembering:

• Standard mode is the default, and it is what instances using Performance Insights fell back to automatically, keeping their existing retention period.
• The Performance Insights API did not change. CloudFormation templates, Terraform configurations, and scripts that set PerformanceInsightsEnabled and PerformanceInsightsRetentionPeriod keep working exactly as written.
• Retention is still 7 days by default at no extra cost, or 1 to 24 months on a paid tier.
• Execution plans and on-demand analysis now require Advanced mode.

For the exam, treat "Performance Insights" and "the DB load view in Database Insights" as the same answer. A question that describes finding the top SQL statement behind a load spike is pointing at this layer either way.

Proactive recommendations

Performance Insights watches selected metrics, learns a threshold from that specific resource's own baseline, and raises a proactive recommendation when values cross it for long enough. The point is to catch a problem while it is still developing rather than after it pages you.

Recommendations appear in the RDS console, either on the account-wide Recommendations page sorted by severity, or on the Recommendations tab of a single database. Each one gives you the detected issue, graphs of the metric against its learned threshold, and an analysis explaining the suggested action. You act on it or dismiss it.

One requirement that gets tested: proactive recommendations need a paid retention tier. The free 7-day retention is not enough for the feature to build a baseline, so an instance on default retention produces none.

Alarming on the right layer

Alarms behave differently per layer, and this is where the layers stop being an academic distinction.

Layer 1 is straightforward. AWS/RDS metrics are ordinary CloudWatch metrics, so an alarm is a normal alarm:

aws cloudwatch put-metric-alarm \
  --alarm-name payments-prod-low-storage \
  --namespace AWS/RDS \
  --metric-name FreeStorageSpace \
  --dimensions Name=DBInstanceIdentifier,Value=payments-prod \
  --statistic Average \
  --period 300 \
  --evaluation-periods 2 \
  --threshold 10737418240 \
  --comparison-operator LessThanThreshold \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:dba-oncall

Layer 2 needs a CloudWatch Logs metric filter first, because Enhanced Monitoring produces log events.

Layer 3 is split. The DBLoad, DBLoadCPU, DBLoadNonCPU, and DBLoadRelativeToNumVCPUs metrics are published to CloudWatch in the AWS/RDS namespace, so you alarm on them normally. Every other Performance Insights counter metric is not published directly, and you reach it with the DB_PERF_INSIGHTS metric math function, which turns it into a time series you can graph and alarm on, including sub-minute high-resolution alarms.

One behavior to expect: the DBLoad metrics are only published when there is load on the instance. Gaps in the graph on a quiet database are normal, so configure how your alarm treats missing data rather than reading a gap as a failure.

Which layer answers which symptom

Exam tips

• "All CloudWatch metrics look normal but the database is slow" is a Performance Insights question every time. Resource metrics cannot see waiting.
• "Process-level" or "per-process CPU and memory on the DB instance" points at Enhanced Monitoring, and only Enhanced Monitoring. You cannot install the CloudWatch agent on an RDS instance, so any option offering to do so is wrong on its face.
• Enhanced Monitoring goes to CloudWatch Logs. If an answer says its metrics appear directly in CloudWatch metrics, eliminate it.
• Watch the granularity numbers: CloudWatch metrics at 60 seconds, Enhanced Monitoring at 1 to 60 seconds, Performance Insights sampling at 1 second.
• Max vCPU is a reference line, not a hard limit. DB load above it means queueing, and only the CPU-versus-non-CPU split tells you whether a bigger instance is the answer.
• BurstBalance is the gp2 volume. EBSIOBalance% and EBSByteBalance% are the DB instance. Different buckets, different fixes.
• Proactive recommendations require paid retention. Free 7-day retention gets you the dashboard, not the recommendations.
• CPUCreditBalance only exists on db.t classes and is published at 5-minute frequency, so a 1-minute alarm period on it will not behave the way you expect.

The one thing to retain: choose the monitoring layer by where the answer lives, not by which console you opened first. Resource metrics tell you the machine is fine; DB load tells you the database is not. The next lesson takes the most common cause of that gap, sessions piling up faster than the database can serve them, and fixes it with connection pooling and a short list of tuning levers.