[RESOLVED] Connectivity Issues (Oregon)

Between 3:55 AM and 4:15 AM PDT, we experienced connectivity issues that impacted connectivity to the US-WEST-2 Region. This impacted multiple AWS services in the Region. Some customers may have also experienced issues accessing the AWS Management Console, with connection timeouts and unresponsive pages. Connectivity within the Region was not affected. Our engineers were automatically engaged at 4:01 AM PDT, and immediately began investigating this issue. We identified the root cause as an issue with networking devices responsible for network routing from the Region to the Seattle Metro, and began working in parallel on multiple paths to mitigate the impact. We took mitigation measures that led to initial recovery at 4:15 AM PDT. As the network continued to stabilize following our mitigation actions, a brief reconvergence event occurred between 4:47 AM and 4:59 AM PDT. During this reconvergence period, some customers may have experienced intermittent connectivity issues to the Region as network routes were re-established. By 4:59 AM PDT, all routes had been fully restored and service metrics returned to pre-event levels. Customers using AWS Direct Connect through EqSe2, Westin Building Exchange, Seattle experienced an extended impact window from 3:55 AM to 5:12 AM PDT. These customers would have experienced connectivity issues until network routes for this specific path were fully restored at 5:12 AM PDT. Customers connected redundantly through other AWS Direct Connect locations were not impacted by this event. The issue has been resolved and all AWS services are operating normally.

We continue to see significant signs of recovery as a result of our mitigation efforts for the connectivity issues impacting multiple AWS services in the US-WEST-2 Region. We have identified the root cause as an issue with a networking device responsible for network routing from the Region to the Seattle Metro. Engineers have finished all mitigation work. As routes continue to be restored, customers should see a continued reduction in error rates and timeouts when connecting to affected services. We are monitoring recovery progress closely and will continue working until all routes have been fully restored and service metrics return to pre-event levels. We will provide another update in the next 30-45 minutes.

We are seeing initial signs of recovery and continue to work toward full recovery.

We are investigating connecivity issues impacting multiple AWS services in the US-WEST-2 Region.

We are investigating connectivity issues impacting multiple AWS services in the US-WEST-2 Region.