The Open S3 Bucket Was a Symptom: How a Missing IAM Condition Caused a Breach
Archi4 min read
Contents
- The alert that started it
- What the logs showed
- The root cause: a trust policy with no conditions
- Why the public bucket was a distraction
- What a durable fix looks like
- 1. Scope the trust policy to a source
- 2. Split roles by pipeline and data class
- 3. Require short sessions and re-authentication
- 4. Alert on role assumption, not just bucket access
- The takeaway for incident reviews
The alert that started it#
At 02:14 on a Tuesday, a cloud security monitor flagged an S3 bucket that allowed public listing. The bucket held customer support exports: ticket text, email addresses, and order numbers for about 340,000 accounts. The on-call engineer rotated the bucket policy within 11 minutes. The incident looked contained.
It was not. The public bucket was the visible failure. The root cause sat two services away, in an IAM trust policy that granted a role to any principal in the same account. That role had read access to the bucket, and a compromised build agent had assumed it weeks earlier.
This is a postmortem of the technical chain, not the people. The goal is to show where the design failed and what a durable fix looks like.
What the logs showed#
CloudTrail told a clean story once analysts lined up the timestamps:
, Day 0, 09:41: A developer added a new CI runner to the account. The runner used an instance profile mapped to a role named ci-deploy-read., Day 0, 09:44: The role's trust policy was edited to allow arn:aws:iam::123456789012:root, which in practice means any principal in the account that is allowed to call sts:AssumeRole. No condition restricted the source., Day 6, 03:12: A dependency in the runner's build script fetched a package from a typosquatted registry. The package ran a postinstall script that called the instance metadata service and retrieved temporary credentials for ci-deploy-read., Day 6, 03:19: The attacker used those credentials to list buckets and found the support export bucket. The bucket policy allowed s3:GetObject to the role., Day 6, 03:20 to Day 9, 18:00: Roughly 4.2 GB of objects were read. No writes, no deletes. The access pattern looked like a backup job to the default alerting rules.
When the public bucket was flagged on Day 9, the responder fixed the bucket policy. The role and its trust relationship stayed in place. The attacker kept reading for another 14 hours until a second alert fired on an unusual user agent string.
The root cause: a trust policy with no conditions#
The failure is not that a bucket was public. The failure is that a role with broad read access could be assumed by anything inside the account, with no source restriction, no session tagging, and no separation between build workloads.
Three design decisions combined:
- Account-level trust. The trust policy used the account root principal. That is a common shortcut. It means any identity in the account with
sts:AssumeRolepermission can take the role, including a compromised CI runner, a developer laptop, or a lambda function. - One role for many jobs.
ci-deploy-readwas used by six pipelines. It had read access to every bucket in the account because nobody wanted to maintain per-pipeline roles. The blast radius was the whole data estate. - No session policy. The role's permissions were static. There was no
aws:SourceVpc, noaws:PrincipalTag, and no requirement for MFA or a signed request. The role could be assumed from anywhere the credentials reached.
Why the public bucket was a distraction#
The public bucket policy was a misconfiguration, but it was not the entry point. The attacker never used it. The bucket was public for listing only; object reads still required IAM permission. The responder fixed the wrong control first because the alert pointed at the most obvious symptom.
This pattern repeats in incident reviews. The loudest alert gets the attention. The quiet trust relationship gets ignored.
What a durable fix looks like#
1. Scope the trust policy to a source#
Replace the account root principal with the specific role or service that needs to assume the role. Add conditions that pin the source:
, aws:SourceVpc or aws:SourceVpce when the caller runs inside a known network., aws:PrincipalTag/team when roles are tagged at creation., sts:ExternalId for third-party integrations.
If a condition cannot be written, the role probably should not exist in its current form.
2. Split roles by pipeline and data class#
One role per pipeline is more work to maintain, but it turns a single compromised runner into a single compromised data set. Tag each role with the pipeline name and the data classification it can reach. Use permission boundaries to cap what any role can do even if its policy is wrong.
3. Require short sessions and re-authentication#
Set the maximum session duration to the minimum the job needs. For build jobs, 15 minutes is often enough. For human access, require MFA and issue sessions of one hour or less. Long-lived sessions are what let the attacker keep reading after the first alert.
4. Alert on role assumption, not just bucket access#
Most teams alert on S3 GetObject volume. Few alert on AssumeRole calls from new source IPs or new user agents. Add a rule for any assumption of a privileged role from an IP that has not used it in the past 30 days. That single rule would have caught this incident on Day 6.
The takeaway for incident reviews#
When a breach is traced, resist the pull of the first misconfiguration you find. Ask what permission made the misconfiguration useful. In this case, the public bucket was noise. The trust policy was the breach.
The fix took four days: 41 roles were rewritten with scoped trust policies, 12 were deleted, and the CI runners were moved to per-pipeline roles with 20-minute sessions. The next red team exercise found the same bucket still public for listing, but no role could read it from outside the build VPC. That is the difference between fixing a symptom and fixing a cause.
More articles