AWS sa-east-1 (São Paulo, Brazil) Geolocation Offline
Incident Report for Ghost Inspector
Postmortem

The root cause of this incident was an outage at the AWS level. Because the outage was region-wide, no remediation was possible on our end. We needed to wait for AWS to bring the region back online. Because this only affected traffic for tests running through the São Paulo, Brazil geolocation, the impact was minimal. After this incident, we considered the idea of rolling traffic over to a different geolocation in this situation, but determined that it is best to let the tests fail and make it clear that the geolocation is having issues. We believe that’s a more useful course of action than redirecting traffic in an unexpected way.

Posted Aug 17, 2021 - 08:36 PDT

Resolved
The AWS sa-east-1 data center appears to be stable again. We are closing this incident.
Posted Oct 10, 2020 - 15:55 PDT
Monitoring
The AWS sa-east-1 (São Paulo, Brazil) region appears to be back online and test traffic is flowing properly. We will continue to monitor the issue.
Posted Oct 10, 2020 - 15:07 PDT
Identified
It appears that the AWS sa-east-1 data center which hosts our São Paulo, Brazil geolocation is currently down. We have confirmed that other AWS customers are experiencing the outage. We will be monitoring the status of the region and will post updates.
Posted Oct 10, 2020 - 14:53 PDT
This incident affected: Test Runner.