Elevated 5xx error rate for connection creation in us-east-1

Incident Report for Ably

Resolved

There was an elevated 5xx error rate for connection creation from the us-east-1 datacenter from 13:24-14:20 UTC, due to a broken router process, which was then removed from service. The error rate in that region peaked at 4%. No other regions were affected.

Any customers using an Ably SDK will not have noticed anything other than potentially increased latencies, since Ably SDKs automatically retry to a fallback region on 5xx. However, customers in east coast or central US who are using SSE, or other protocols using our protocol translation functionality (such as mqtt, pusher, or pubnub), may have noticed some connection attempts fail with 5xx.

Posted Jun 18, 2024 - 13:24 UTC