Incident with Webhooks
GitHub Status · Resolved
Resolved
Unknown
Jun 26, 2026 18:35 UTC
Jun 11, 2026 22:21 UTC
Timeline
Resolved
On June 11, 2026, between 19:28 UTC and 21:06 UTC, GitHub webhook deliveries were delayed. Average delivery latency peaked at approximately 3.4 minutes, with some deliveries delayed by as much as 62 minutes at the 99th percentile. No events were lost — delayed events were queued and delivered once processing caught up.This was due to a change in how webhook traffic was distributed across regions: to relieve load on one region, a portion of processing was shifted to another, where higher latency prevented our delivery workers from keeping pace with incoming volume, creating a backlog. We mitigated the incident by rebalancing webhook traffic distribution; as load returned to normal levels, processing caught up and the delivery backlog fully drained.We are working on improving the accuracy of the utilization metrics used to scale our delivery worker pools, and reassess how we distribute webhook traffic across regions, to reduce our time to detection and mitigation of issues like this one in the future.
Resolved
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.
Monitoring
The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability.
Monitoring
The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability.
Update
We have applied a mitigation and are monitoring for recovery.
Update
We have applied a mitigation and are monitoring for recovery.
Update
We have applied a mitigation and are monitoring for recovery.
Investigating
We are investigating reports of degraded performance for Webhooks
Update
We are currently experiencing delays in Web Hook delivery and are actively investigating the root cause.
Investigating
We are investigating reports of degraded performance for Webhooks
Update
We are currently experiencing delays in Web Hook delivery and are actively investigating the root cause.
Investigating
We are investigating reports of degraded performance for Webhooks
Update
We are currently experiencing delays in Web Hook delivery and are actively investigating the root cause.