Incident with Webhooks

GitHub Status · Resolved

Status

Resolved

Severity

Unknown

Started

Jun 26, 2026 18:35 UTC

Resolved

Jun 11, 2026 22:21 UTC

Timeline

Resolved · GitHub Status · Jun 11 22:19 UTC

Resolved

On June 11, 2026, between 19:28 UTC and 21:06 UTC, GitHub webhook deliveries were delayed. Average delivery latency peaked at approximately 3.4 minutes, with some deliveries delayed by as much as 62 minutes at the 99th percentile. No events were lost — delayed events were queued and delivered once processing caught up.This was due to a change in how webhook traffic was distributed across regions: to relieve load on one region, a portion of processing was shifted to another, where higher latency prevented our delivery workers from keeping pace with incoming volume, creating a backlog. We mitigated the incident by rebalancing webhook traffic distribution; as load returned to normal levels, processing caught up and the delivery backlog fully drained.We are working on improving the accuracy of the utilization metrics used to scale our delivery worker pools, and reassess how we distribute webhook traffic across regions, to reduce our time to detection and mitigation of issues like this one in the future.

Incident with Webhooks →
Resolved · GitHub Status · Jun 11 22:19 UTC

Resolved

This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

Incident with Webhooks →
Monitoring · GitHub Status · Jun 11 20:00 UTC

Monitoring

The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability.

Incident with Webhooks →
Monitoring · GitHub Status · Jun 11 20:00 UTC

Monitoring

The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability.

Incident with Webhooks →
Update · GitHub Status · Jun 11 19:58 UTC

Update

We have applied a mitigation and are monitoring for recovery.

Incident with Webhooks →
Update · GitHub Status · Jun 11 19:58 UTC

Update

We have applied a mitigation and are monitoring for recovery.

Incident with Webhooks →
Update · GitHub Status · Jun 11 19:58 UTC

Update

We have applied a mitigation and are monitoring for recovery.

Incident with Webhooks →
Investigating · GitHub Status · Jun 11 19:42 UTC

Investigating

We are investigating reports of degraded performance for Webhooks

Incident with Webhooks →
Update · GitHub Status · Jun 11 19:42 UTC

Update

We are currently experiencing delays in Web Hook delivery and are actively investigating the root cause.

Incident with Webhooks →
Investigating · GitHub Status · Jun 11 19:42 UTC

Investigating

We are investigating reports of degraded performance for Webhooks

Incident with Webhooks →
Update · GitHub Status · Jun 11 19:42 UTC

Update

We are currently experiencing delays in Web Hook delivery and are actively investigating the root cause.

Incident with Webhooks →
Investigating · GitHub Status · Jun 11 19:42 UTC

Investigating

We are investigating reports of degraded performance for Webhooks

Incident with Webhooks →
Update · GitHub Status · Jun 11 19:42 UTC

Update

We are currently experiencing delays in Web Hook delivery and are actively investigating the root cause.

Incident with Webhooks →