Problems with US-only routing policies

Incident Report for Ably

Resolved

Customers using a US-only routing policy (eg environment="us") who would naturally have been routed to us-east-1 may be experiencing issues, we are investigating

19th Jul 11:04 AM
This has now been resolved. The cause was incorrect DNS configuration for the US-only routing policy. We had taken the us-east-1 region down for maintenance; the command to do this updates all relevant routing policies, in this case while the global DNS routing policy was updated correctly, the US-only routing policy was not. We apologise for the inconvenience and will be taking steps to make sure this does not happen again.

Ably SDKs will automatically retry any failed attempts to a fallback site, and this mechanism was still working as normal. So in most cases, the only customer-visible effect to those affected (those using a US-only routing policy who are nearer us-east-1 than us-east-2 or us-west-1) will have been higher latencies when establishing a connection or making a REST request.
Posted Jul 19, 2023 - 10:35 UTC