Elevated Error Rates ingesting Insights and API Analytics Data
Incident Report for US1 - Anypoint Platform
Resolved
We have resolved the issue. We will be providing an RCA via our support portal in the next 2 business days. During this incident, applications and API Gateways continued to run without interruptions.
Posted 5 months ago. May 30, 2018 - 15:36 PDT
Monitoring
The services are back online. We are investigating the consequences of the service outage and monitoring recovery.
Posted 5 months ago. May 30, 2018 - 15:06 PDT
Update
We are moving Analytics to Operational State and will be monitoring this service for the next hour. We are currently in the last phases of validation around Object Store and will move this entire incident to Monitoring shortly,
Posted 5 months ago. May 30, 2018 - 12:00 PDT
Update
We have changed the status for Analytics from Partial Outage to Degraded Performance. Both Analytics and Object Store are on the path to recovery and we will post in 30 minutes further status.
Posted 5 months ago. May 30, 2018 - 11:39 PDT
Update
The engineering teams are still working on the affected systems for both Analytics and Object Store. We have seen an improvement in both services and are still working on the final resolution. We will have another update in 1 hour, which is pending an additional set of changes.
Posted 5 months ago. May 30, 2018 - 08:38 PDT
Update
Enhanced Logging service has returned to normal. We continue to work on restoring full functionality for Analytics and Object Store. We will have an update posted in 30 minutes.
Posted 5 months ago. May 30, 2018 - 07:58 PDT
Update
We are also investigating issues with Enhanced logging, We will be posting an update shortly as the team is currently investigating.
Posted 5 months ago. May 30, 2018 - 05:30 PDT
Identified
We identified additional errors on API Analytics data ingest and experience increased error rate on Object Store. The engineering team is working on resolving the issues.
Posted 5 months ago. May 30, 2018 - 02:55 PDT
Monitoring
Insights service is back online and processing events in normal capacity. API Analytics is online but still processing queued events. Object Store continues to work normally. We are monitoring recovery.
Posted 5 months ago. May 29, 2018 - 21:08 PDT
Update
Object Store Service has been restored. API Analytics Service has been restored, there are still some delays on processing the queued data . Insights Analytics still has high error rates.
Posted 5 months ago. May 29, 2018 - 18:56 PDT
Update
API Analytics service has been restored and it's catching up with the data ingestion. We are still working on Insights Data ingestion. It also affected the usage of the Object Store in CloudHub, causing elevated errors on Object Store API.
Posted 5 months ago. May 29, 2018 - 16:21 PDT
Identified
We have confirmed elevated error rate with Insights and API Analytics data ingress and have identified the issue and expect to have it resolved soon.
Posted 5 months ago. May 29, 2018 - 14:40 PDT
Investigating
We are experiencing elevated error rates with Insights and API Analytics data ingress. Our engineers are investigating.
Posted 5 months ago. May 29, 2018 - 14:14 PDT
This incident affected: Runtime Services (Object Store, Insights) and Anypoint Management Center (API Analytics).