ChatGPT, API and Sora outage: a new telemetry service overwhelms Kubernetes
A newly deployed monitoring service made every node in OpenAI's clusters hammer the Kubernetes API, taking down cluster control planes and the DNS that services use to find each other, for over four hours.
- Security:
- Explicitly ruled out
- Started
- Dec 11, 2024, 23:16 UTC
- All services fully recovered
- Dec 12, 2024, 03:38 UTC
- Duration
- 4 h 22 min
- Full recovery
- Complete
- Geographic scope
- Global
- Last verified
- 1 source
What happened
- Systems and services affected
- ChatGPTOpenAI APISora
- Kind of failure
- Configuration change
- Sector
- AI services
- Provider report
- Published
- Status and recovery
- Resolved. OpenAI reports substantial recovery for the API at 5:36 PM PST and ChatGPT at 5:45 PM, ChatGPT and Sora fully recovered at 7:01 PM, and the API across all models at 7:38 PM.[1]
OpenAI gives times in PST: 3:16 PM to 7:38 PM on December 11.
What caused it?
Root cause: ConfirmedA new telemetry service deployed at 3:12 PM PST had a configuration that made every node in each cluster run resource-intensive Kubernetes API operations. The API servers were overwhelmed, taking down the control plane in most large clusters; DNS-based service discovery depends on the control plane, so services could no longer find each other.[1]
Provider report: Published. OpenAI published a write-up on its status page. The page does not show its publication date.[1]
Was it malicious?
Explicitly ruled outOpenAI: the event was "not caused by a security incident or a recent launch."[1]
How the failure happened
- 1TriggerNew telemetry service deployed
- 2FailureKubernetes API servers overwhelmed
- 3SpreadControl plane down; service discovery failed
- Affected3 services and systems
Evidence for each step
- 1
- 2Kubernetes API servers overwhelmed. Every node ran resource-intensive API operations[1]Led from step 1: Stated in a provider report or official record.
- 3Control plane down; service discovery failed. DNS-based service discovery depends on the control plane[1]Led from step 2: Stated in a provider report or official record.
Only steps the sources state are drawn. Affected services are listed under What happened.
Explanation
Both versions summarize the sourced claims on this page. Our own interpretation appears only under Independent analysis.
Plain language: what happened and why people were affected.
On December 11, 2024, ChatGPT, OpenAI's API and Sora stopped working for more than four hours.
OpenAI rolled out a new tool to monitor its computing clusters. Its settings made every machine send heavy requests to the system that manages each cluster, which buckled under the load.
The fix meant removing the new tool, but engineers needed that same overloaded management system to do it, which slowed recovery.
Confirmed and unknown
Confirmed
- ConfirmedA new telemetry service's configuration made every node run resource-intensive Kubernetes API operations, overwhelming the API servers.[1]Said by: OpenAI · Evidence: Official
- ConfirmedDNS caching made the issue less visible at first, so the rollout continued.[1]Said by: OpenAI · Evidence: Official
- ConfirmedFixing it required control-plane access that the overload had blocked.[1]Said by: OpenAI · Evidence: Official
Denied
- DeniedIt was caused by a security incident or a product launch.[1]Said by: OpenAI · Evidence: Official
Still unknown
No open questions: the provider has published a full account.
Timeline
Times in UTC. Dates without a time mean the sources give no time. Only events a source documents are listed.
- First observedDec 11, 2024, 23:16 UTC
- AcknowledgedNot documented
- MitigationDec 12, 2024, 01:36 UTC
- RecoveryDec 12, 2024, 01:45 UTC
- ResolvedDec 12, 2024, 03:38 UTC
- Incident reportPublished, date not shown
Evidence and sources
Grouped by type of evidence. Numbers match the citations above.
Primary: provider and official sources
Published by the provider or agency itself: incident reports, status records, advisories, statements.
- 1.API, ChatGPT & Sora Facing Issues (write-up) (opens in a new tab)OpenAI · Provider report · Publication date not recorded · Accessed
Last verified against these sources on Nov 14, 2026.