GitHub outage: a 43-second network break leaves the site degraded for 24 hours
A brief loss of connectivity during network maintenance caused GitHub's databases to fail over across the country, leaving data out of sync and the site degraded for 24 hours and 11 minutes.
- Security:
- No determination
- Started
- Oct 21, 2018, 22:52 UTC
- Status returned to green
- Oct 22, 2018, 23:03 UTC
- Duration
- 24 h 11 min
- Full recovery
- Complete
- Geographic scope
- Global
- Last verified
- 1 source
What happened
- Systems and services affected
- GitHub.comWebhooksGitHub Pages
- Kind of failure
- Network equipment
- Sector
- Software development
- Provider report
- Published
- Status and recovery
- Resolved. GitHub processed all pending webhooks and Pages builds and set its status to green at 23:03 UTC on October 22. Reconciling writes that had not replicated was still in progress when GitHub published its analysis.[1]
What caused it?
Root cause: ConfirmedRoutine maintenance to replace failing 100G optical equipment caused a 43-second loss of connectivity between GitHub's US East Coast network hub and its primary US East Coast data center. Database failover automation (Orchestrator) promoted West Coast databases, and writes that had not replicated had to be reconciled.[1]
Provider report: Published. GitHub published its post-incident analysis on October 30, 2018.[1]
Was it malicious?
No determinationGitHub attributes the incident to maintenance work and a network partition. Its analysis does not mention an attack.[1]
How the failure happened
- 1TriggerMaintenance on failing optical equipment
- 2Failure43-second loss of connectivity
- 3ResponseDatabases failed over to the West Coast
- Affected3 services and systems
Evidence for each step
- 1
- 243-second loss of connectivity. Between the US East Coast hub and the primary US East Coast data center[1]Led from step 1: Stated in a provider report or official record.
- 3Databases failed over to the West Coast. Writes that had not replicated had to be reconciled[1]Led from step 2: Stated in a provider report or official record.
Only steps the sources state are drawn. Affected services are listed under What happened.
Explanation
Both versions summarize the sourced claims on this page. Our own interpretation appears only under Independent analysis.
Plain language: what happened and why people were affected.
On October 21, 2018, GitHub stopped showing up-to-date information and paused some features for about a day.
During maintenance, the connection between GitHub's main East Coast data center and its network was cut for 43 seconds. In that time, automatic systems moved the main databases to the West Coast. When the connection came back, the two coasts had different data.
GitHub chose to keep the site degraded rather than risk losing data while it put everything back in order. It says no user data was lost.
Confirmed and unknown
Confirmed
- ConfirmedConnectivity between the East Coast hub and the primary data center was lost for 43 seconds during maintenance on failing 100G optical equipment.[1]Said by: GitHub · Evidence: Official
- ConfirmedGitHub had degraded service for 24 hours and 11 minutes.[1]Said by: GitHub · Evidence: Official
- ConfirmedNo user data was lost.[1]Said by: GitHub · Evidence: Official
- ConfirmedAbout 200,000 webhook payloads that had outlived an internal TTL were dropped.[1]Said by: GitHub · Evidence: Official
Still unknown
No open questions: the provider has published a full account.
Timeline
Times in UTC. Dates without a time mean the sources give no time. Only events a source documents are listed.
- First observedOct 21, 2018, 22:52 UTC
- AcknowledgedOct 21, 2018, 23:09 UTC
- MitigationOct 21, 2018, 23:19 UTC
- RecoveryOct 22, 2018, 16:24 UTC
- ResolvedOct 22, 2018, 23:03 UTC
- Incident reportOct 30, 2018
- First observed43-second connectivity loss; database clusters fail over to the West Coast.[1]
- AcknowledgedGitHub sets its status to yellow, then red at 23:11.[1]
- MitigationWebhook delivery and Pages builds paused to protect data integrity.[1]
- All database primaries re-established on the East Coast.[1]
- RecoveryFailover back to the original topology; backlog processing continues.[1]
- ResolvedPending webhooks and Pages builds processed; status set to green.[1]
- After the incident endedIncident reportGitHub publishes its post-incident analysis.[1]
- Start to: Status returned to green24 h 11 min
Evidence and sources
Grouped by type of evidence. Numbers match the citations above.
Primary: provider and official sources
Published by the provider or agency itself: incident reports, status records, advisories, statements.
- 1.October 21 post-incident analysis (opens in a new tab)GitHub · Provider report · Published · Accessed
Last verified against these sources on Nov 14, 2026.