Google Cloud·

Google Cloud outage: a blank policy field crashes the API control system

Unexpected blank fields in a policy update hit an unprotected code path in Google's Service Control, crashing it globally and causing 503 errors across dozens of Google Cloud products.

ResolvedEnded Jun 13, 2025
Security:
No determination
Started
Jun 12, 2025, 17:51 UTC
Incident end on Google's status page
Jun 13, 2025, 01:18 UTC
Duration
7 h 27 min
Full recovery
Complete
Geographic scope
Global
Last verified
1 source

What happened

Systems and services affected
Google Cloud APIs (dozens of products incl. Compute Engine, Cloud Storage, BigQuery, IAM, Cloud Run)Some Google Workspace apps
Kind of failure
Software defect
Sector
Cloud infrastructure
Provider report
Published
Status and recovery
Resolved. Google's status page marks the incident over at 18:18 PDT. Most regions recovered within about three hours; us-central1 took longer.[1]

Google's status page gives 10:51 to 18:18 PDT. Most regions recovered within about three hours; the end time covers the long tail.

What caused it?

Root cause: Confirmed

A new quota policy check added to Service Control on May 29 had no error handling and no feature flag. On June 12 a policy change with unintended blank fields reached that code, causing a null pointer crash loop worldwide.[1]

Provider report: Published. Google published an incident report on June 13, 2025.[1]

Was it malicious?

No determination

Google's report attributes the outage to an internal software defect and a faulty policy change. It does not mention an attack.[1]

How the failure happened

  1. 1TriggerNew quota check (May 29)
  2. 2FailurePolicy change with blank fields (Jun 12)
  3. 3SpreadCrash loop worldwide
  4. Affected2 services and systems

Evidence for each step

  1. 1
    New quota check (May 29). No error handling and no feature flag[1]
    Starting point.
  2. 2
    Policy change with blank fields (Jun 12). Replicated globally within seconds[1]
    Led from step 1: Stated in a provider report or official record.
  3. 3
    Crash loop worldwide. Null pointer crash in Service Control; restarts overloaded Spanner in us-central1[1]
    Led from step 2: Stated in a provider report or official record.

Only steps the sources state are drawn. Affected services are listed under What happened.

Explanation

Both versions summarize the sourced claims on this page. Our own interpretation appears only under Independent analysis.

Plain language: what happened and why people were affected.

On June 12, 2025, many Google Cloud services returned errors for several hours, and apps built on them failed too.

Every request to Google Cloud passes through a system that checks permissions and limits. New code in that system had a hidden flaw. When a settings update with some blank fields arrived, the code crashed, over and over, everywhere at once.

Google switched the new code off in about 40 minutes. Its largest US region took longer because all the restarting servers hit the same database at once.

Confirmed and unknown

Confirmed

  • Confirmed
    A new Service Control feature added on May 29 lacked error handling and feature-flag protection.[1]
    Said by: Google · Evidence: Official
  • Confirmed
    A policy change with unintended blank fields replicated globally within seconds and triggered a null pointer crash loop.[1]
    Said by: Google · Evidence: Official
  • Confirmed
    Recovery in us-central1 was slowed by restarting tasks overloading Spanner, because exponential backoff was missing.[1]
    Said by: Google · Evidence: Official

Still unknown

No open questions: the provider has published a full account.

Timeline

Times in UTC. Dates without a time mean the sources give no time. Only events a source documents are listed.

  1. First observedJun 12, 2025, 17:51 UTC
  2. AcknowledgedNot documented
  3. MitigationNot documented
  4. RecoveryNot documented
  5. ResolvedJun 13, 2025, 01:18 UTC
  6. Incident reportJun 13, 2025
Jun 12, 2025, 17:45 UTCJun 13, 2025, 01:18 UTC
  1. New quota policy check added to Service Control.[1]
  2. Policy change with blank fields inserted into Spanner and replicated globally.[1]
  3. First observed
    Incident start on Google's status page; 503 errors across products.[1]
  4. Resolved
    Incident end on Google's status page.[1]
  5. Incident report
    Google publishes its incident report.[1]
  6. Start to: Incident end on Google's status page7 h 27 min

Evidence and sources

Grouped by type of evidence. Numbers match the citations above.

Primary: provider and official sources

Published by the provider or agency itself: incident reports, status records, advisories, statements.

  1. 1.
    Multiple GCP products are experiencing Service issues (Incident Report) (opens in a new tab)
    Google Cloud · Provider report · Published · Accessed

Last verified against these sources on Nov 14, 2026.