We’ve confirmed that the solution implemented to mitigate the issue causing the disruption was successful. Engineers have observed that the service is now stable and have declared the incident resolved. Customers should no longer see any impact. We will perform a full investigation of the incident, confirming the technical trigger, the underlying cause, and preventive action to avoid a repeat in the future. We apologize for how you and your business may have been affected by this incident.
Posted Aug 22, 2024 - 16:42 PDT
Monitoring
We have completed all required work streams and implemented a fix. Monitoring and validation of stability and performance levels have now begun. We’ll provide an update in 60 minutes or sooner if additional information becomes available. Customers might experienced increased response times for publish and errors around 0.08% starting at around 7:15 PT (14:15 UTC) Client has exponential retries in place for 502 and 429 errors, as of 16:15 PT (1:15 UTC) customers should experience better performance while working with the Anypoint MQ broker in us-east-1
Posted Aug 22, 2024 - 16:23 PDT
Update
We continue to work on this issue, MQ Broker seeing increased response times for publish and errors. About 0.08% of the requests are failing with 502 errors because of timeouts. Client has exponential retries in place for 502 errors.