Slack outage after the holidays: overloaded AWS network gateways

As people returned from the holidays, an AWS Transit Gateway in Slack's network became overloaded and dropped packets, and Slack's own scaling and monitoring systems made recovery harder.

ResolvedEnded Jan 4, 2021
Security:
No determination
Started
Jan 4, 2021
Network, errors and latency back to normal
Jan 4, 2021, 18:40 UTC
Duration
Not computable from sources
Full recovery
Complete
Geographic scope
Global
Last verified
1 source

What happened

Systems and services affected
Slack messaging and web tierSlack's internal provisioning serviceSlack's dashboards and alerting
Kind of failure
Network equipment
Sector
Workplace software
Provider report
Published
Status and recovery
Resolved. At 10:40 AM PST AWS's added Transit Gateway capacity had reached all Availability Zones and Slack's network, error rates and latency returned to normal.[1]

Slack's report gives times in PST and no exact start time; error rates rose during the Americas' morning.

What caused it?

Root cause: Confirmed

An AWS Transit Gateway in Slack's network became overloaded as traffic jumped after the holidays and dropped packets. AWS told Slack that part of its networking infrastructure "had indeed become saturated and was dropping packets." Slack's autoscaling and instance provisioning then ran into limits that slowed recovery.[1]

Provider report: Published. Slack published its report on February 1, 2021.[1]

Was it malicious?

No determination

Slack attributes the outage to network overload and packet loss. Its report does not mention an attack.[1]

How the failure happened

  1. 1TriggerTraffic jump after the holidays
  2. 2FailureAWS Transit Gateway saturated
  3. 3SpreadRecovery slowed
  4. Affected3 services and systems

Evidence for each step

  1. 1
    Traffic jump after the holidays[1]
    Starting point.
  2. 2
    AWS Transit Gateway saturated. Dropping packets[1]
    Led from step 1: Stated in a provider report or official record.
  3. 3
    Recovery slowed. Provisioning hit file and quota limits; monitoring shared the same gateways[1]
    Led from step 2: Stated in a provider report or official record.

Only steps the sources state are drawn. Affected services are listed under What happened.

Explanation

Both versions summarize the sourced claims on this page. Our own interpretation appears only under Independent analysis.

Plain language: what happened and why people were affected.

On January 4, 2021, as people returned to work after the holidays, Slack stopped working for several hours.

Slack runs on Amazon Web Services. Part of the network that connects Slack's systems inside AWS got overloaded when everyone came back at once, and started losing data in transit.

Slack's automatic systems then tried to add a large number of servers at once and hit limits, which slowed the recovery.

Confirmed and unknown

Confirmed

  • Confirmed
    One of Slack's AWS Transit Gateways became overloaded and dropped packets.[1]
    Said by: Slack, citing AWS · Evidence: Official
  • Confirmed
    Slack's provisioning service hit the Linux open-files limit and an AWS quota limit while adding servers.[1]
    Said by: Slack · Evidence: Official
  • Confirmed
    Slack's dashboards and alerting depended on the same Transit Gateways, so monitoring was impaired.[1]
    Said by: Slack · Evidence: Official
  • Confirmed
    AWS manually increased Transit Gateway capacity and is reviewing how Transit Gateway scales for large jumps in traffic.[1]
    Said by: Slack, citing AWS · Evidence: Official

Still unknown

No open questions: the provider has published a full account.

Timeline

Times in UTC. Dates without a time mean the sources give no time. Only events a source documents are listed.

  1. First observedJan 4, 2021
  2. AcknowledgedNot documented
  3. MitigationJan 4, 2021, 16:15 UTC
  4. RecoveryJan 4, 2021, 17:15 UTC
  5. ResolvedJan 4, 2021, 18:40 UTC
  6. Incident reportFeb 1, 2021
Jan 4, 2021, 15:00 UTCJan 4, 2021, 18:40 UTC
  1. First observed
    Error rates rise during the Americas' morning; an external monitoring service pages Slack's team.[1]
  2. At the 7:00 AM PST traffic peak, the web tier saturates and Slack becomes unavailable.[1]
  3. Mitigation
    Instance provisioning restored.[1]
  4. Recovery
    Slack is "degraded, not down."[1]
  5. Resolved
    Added Transit Gateway capacity reaches all Availability Zones; network back to normal.[1]
  6. After the incident ended
    Incident report
    Slack publishes its report.[1]

Evidence and sources

Grouped by type of evidence. Numbers match the citations above.

Primary: provider and official sources

Published by the provider or agency itself: incident reports, status records, advisories, statements.

  1. 1.
    Slack's Outage on January 4th 2021 (opens in a new tab)
    Slack Engineering · Provider report · Published · Accessed

Last verified against these sources on Nov 14, 2026.