Microsoft cloud outage: one router command disrupts Azure's global network

A command added to a router procedure without retesting purged routing data across Microsoft's wide-area network, disrupting Azure, Microsoft 365 and Power Platform worldwide for over five hours.

ResolvedEnded Jan 25, 2023
Security:
No determination
Started
Jan 25, 2023, 07:08 UTC
Full mitigation
Jan 25, 2023, 12:43 UTC
Duration
5 h 35 min
Full recovery
Complete
Geographic scope
Global
Last verified
1 source

What happened

Systems and services affected
Azure regionsMicrosoft 365Power PlatformAzure Government (dependent services)
Kind of failure
Human error
Sector
Cloud infrastructure
Provider report
Published
Provider incident IDs
VSG1-B90
Status and recovery
Resolved. Nearly all network devices recovered by 09:05 UTC and the last by 09:25 UTC. Some traffic took suboptimal paths until engineers restarted paused WAN automation; full recovery was at 12:43 UTC.[1]

Most regions recovered by 09:05 UTC; intermittent packet loss continued until 12:43 UTC.

What caused it?

Root cause: Confirmed

While adding WAN capacity in Madrid, an engineer ran a standard operating procedure that had been modified without retesting. It included a command to purge the IGP database which, on one vendor's routers, runs across all routers joined to the IGP. All routers recomputed connectivity and BGP routers re-advertised Internet prefixes. The same operation ran on a second Madrid router 33 minutes later.[1]

Provider report: Published. Microsoft published a post-incident review under tracking ID VSG1-B90.[1]

Was it malicious?

No determination

Microsoft attributes the incident to an unqualified network change. Its review does not mention an attack.[1]

How the failure happened

  1. 1TriggerModified procedure, not retested
  2. 2FailureCommand purged the IGP database
  3. 3SpreadAll routers recomputed connectivity
  4. Affected4 services and systems

Evidence for each step

  1. 1
    Modified procedure, not retested. Run while adding WAN capacity in Madrid[1]
    Starting point.
  2. 2
    Command purged the IGP database. On one vendor's routers it runs across all routers joined to the IGP[1]
    Led from step 1: Stated in a provider report or official record.
  3. 3
    All routers recomputed connectivity. BGP routers re-advertised Internet prefixes; a second Madrid router repeated it 33 minutes later[1]
    Led from step 2: Stated in a provider report or official record.

Only steps the sources state are drawn. Affected services are listed under What happened.

Explanation

Both versions summarize the sourced claims on this page. Our own interpretation appears only under Independent analysis.

Plain language: what happened and why people were affected.

On January 25, 2023, people around the world had trouble reaching Microsoft's cloud, including Microsoft 365, Power Platform and Azure, for several hours.

An engineer making a planned change to one router in Madrid used a procedure that had been edited without being tested. One command in it, on that brand of router, affected every connected router in Microsoft's global network, not just the one being worked on.

Every router had to rebuild its view of the network, and the same step was repeated on a second router 33 minutes later, causing a second wave.

Confirmed and unknown

Confirmed

  • Confirmed
    A standard operating procedure modified without retesting added a command to purge the IGP database.[1]
    Said by: Microsoft · Evidence: Official
  • Confirmed
    On one vendor's routers the command runs across all IGP-joined routers, and it was not on the block list.[1]
    Said by: Microsoft · Evidence: Official
  • Confirmed
    The same operation ran on a second Madrid router 33 minutes later, causing a second wave of impact.[1]
    Said by: Microsoft · Evidence: Official
  • Confirmed
    Impact lasted from 07:08 to 12:43 UTC; most regions recovered by 09:05 UTC.[1]
    Said by: Microsoft · Evidence: Official

Still unknown

No open questions: the provider has published a full account.

Timeline

Times in UTC. Dates without a time mean the sources give no time. Only events a source documents are listed.

  1. First observedJan 25, 2023, 07:08 UTC
  2. AcknowledgedJan 25, 2023, 08:20 UTC
  3. MitigationJan 25, 2023, 09:05 UTC
  4. RecoveryJan 25, 2023, 09:25 UTC
  5. ResolvedJan 25, 2023, 12:43 UTC
  6. Incident reportPublished, date not shown
Jan 25, 2023, 07:08 UTCJan 25, 2023, 12:43 UTC
  1. First observed
    Router command runs; connectivity across Azure regions and to the Internet degrades.[1]
  2. Acknowledged
    The problematic command is identified.[1]
  3. Mitigation
    Nearly all network devices recovered.[1]
  4. Recovery
    Last networking equipment recovered.[1]
  5. Resolved
    Full mitigation after paused WAN automation is restarted.[1]
  6. Start to: Full mitigation5 h 35 min

Evidence and sources

Grouped by type of evidence. Numbers match the citations above.

Primary: provider and official sources

Published by the provider or agency itself: incident reports, status records, advisories, statements.

  1. 1.
    Post Incident Review (PIR): Azure Networking, Global WAN issues (VSG1-B90) (opens in a new tab)
    Microsoft Azure · Provider report · Published · Accessed

Last verified against these sources on Nov 14, 2026.