Google Cloud outage: a blank policy field crashes the API control system
Unexpected blank fields in a policy update hit an unprotected code path in Google's Service Control, crashing it globally and causing 503 errors across dozens of Google Cloud products.
- Security:
- No determination
- Started
- Jun 12, 2025, 17:51 UTC
- Incident end on Google's status page
- Jun 13, 2025, 01:18 UTC
- Duration
- 7 h 27 min
- Full recovery
- Complete
- Geographic scope
- Global
- Last verified
- 1 source
What happened
- Systems and services affected
- Google Cloud APIs (dozens of products incl. Compute Engine, Cloud Storage, BigQuery, IAM, Cloud Run)Some Google Workspace apps
- Kind of failure
- Software defect
- Sector
- Cloud infrastructure
- Provider report
- Published
- Status and recovery
- Resolved. Google's status page marks the incident over at 18:18 PDT. Most regions recovered within about three hours; us-central1 took longer.[1]
Google's status page gives 10:51 to 18:18 PDT. Most regions recovered within about three hours; the end time covers the long tail.
What caused it?
Root cause: ConfirmedA new quota policy check added to Service Control on May 29 had no error handling and no feature flag. On June 12 a policy change with unintended blank fields reached that code, causing a null pointer crash loop worldwide.[1]
Provider report: Published. Google published an incident report on June 13, 2025.[1]
Was it malicious?
No determinationGoogle's report attributes the outage to an internal software defect and a faulty policy change. It does not mention an attack.[1]
How the failure happened
- 1TriggerNew quota check (May 29)
- 2FailurePolicy change with blank fields (Jun 12)
- 3SpreadCrash loop worldwide
- Affected2 services and systems
Evidence for each step
- 1
- 2Policy change with blank fields (Jun 12). Replicated globally within seconds[1]Led from step 1: Stated in a provider report or official record.
- 3Crash loop worldwide. Null pointer crash in Service Control; restarts overloaded Spanner in us-central1[1]Led from step 2: Stated in a provider report or official record.
Only steps the sources state are drawn. Affected services are listed under What happened.
Explanation
Both versions summarize the sourced claims on this page. Our own interpretation appears only under Independent analysis.
Plain language: what happened and why people were affected.
On June 12, 2025, many Google Cloud services returned errors for several hours, and apps built on them failed too.
Every request to Google Cloud passes through a system that checks permissions and limits. New code in that system had a hidden flaw. When a settings update with some blank fields arrived, the code crashed, over and over, everywhere at once.
Google switched the new code off in about 40 minutes. Its largest US region took longer because all the restarting servers hit the same database at once.
Confirmed and unknown
Confirmed
- ConfirmedA new Service Control feature added on May 29 lacked error handling and feature-flag protection.[1]Said by: Google · Evidence: Official
- ConfirmedA policy change with unintended blank fields replicated globally within seconds and triggered a null pointer crash loop.[1]Said by: Google · Evidence: Official
- ConfirmedRecovery in us-central1 was slowed by restarting tasks overloading Spanner, because exponential backoff was missing.[1]Said by: Google · Evidence: Official
Still unknown
No open questions: the provider has published a full account.
Timeline
Times in UTC. Dates without a time mean the sources give no time. Only events a source documents are listed.
- First observedJun 12, 2025, 17:51 UTC
- AcknowledgedNot documented
- MitigationNot documented
- RecoveryNot documented
- ResolvedJun 13, 2025, 01:18 UTC
- Incident reportJun 13, 2025
- New quota policy check added to Service Control.[1]
- Policy change with blank fields inserted into Spanner and replicated globally.[1]
- First observedIncident start on Google's status page; 503 errors across products.[1]
- ResolvedIncident end on Google's status page.[1]
- Incident reportGoogle publishes its incident report.[1]
- Start to: Incident end on Google's status page7 h 27 min
Evidence and sources
Grouped by type of evidence. Numbers match the citations above.
Primary: provider and official sources
Published by the provider or agency itself: incident reports, status records, advisories, statements.
- 1.Multiple GCP products are experiencing Service issues (Incident Report) (opens in a new tab)Google Cloud · Provider report · Published · Accessed
Last verified against these sources on Nov 14, 2026.