Resolved -
Service has remained stable over the past week and thus the incident is resolved. Note that rate limits may dynamically adjust to preserve service health in times of high network activity or volatility.
Mar 18, 23:01 UTC
Update -
Fix appears to have resolved performance issues, but reduced rate limits are still in effect as we continue to monitor.
Mar 12, 18:53 UTC
Update -
A fix has been deployed. We are monitoring to ensure it is stable.
Mar 11, 23:15 UTC
Update -
Additional reduction of rate limits.
Mar 11, 21:00 UTC
Update -
Performance degradation observed again.
Mar 11, 20:00 UTC
Update -
We flipped the standby and active endpoint. This temporarily fixed the problem.
Mar 11, 15:15 UTC
Monitoring -
Reports of degraded performance in Discord.
Mar 11, 15:00 UTC
Update -
We reduced global rate limits by half for Horizon.
Mar 11, 14:30 UTC
Investigating -
We've identified a high CPU utilization on SDF Horizon nodes due to increased request load that saturated DB connections.
Mar 11, 14:00 UTC