We have resolved the issue. We will be providing an RCA via our support portal in the next 2 business days. During this incident, applications and API Gateways continued to run without interruptions.
Posted 23 days ago. May 30, 2018 - 15:36 PDT
The services are back online. We are investigating the consequences of the service outage and monitoring recovery.
Posted 23 days ago. May 30, 2018 - 15:06 PDT
We are moving Analytics to Operational State and will be monitoring this service for the next hour. We are currently in the last phases of validation around Object Store and will move this entire incident to Monitoring shortly,
Posted 23 days ago. May 30, 2018 - 12:00 PDT
We have changed the status for Analytics from Partial Outage to Degraded Performance. Both Analytics and Object Store are on the path to recovery and we will post in 30 minutes further status.
Posted 23 days ago. May 30, 2018 - 11:39 PDT
The engineering teams are still working on the affected systems for both Analytics and Object Store. We have seen an improvement in both services and are still working on the final resolution. We will have another update in 1 hour, which is pending an additional set of changes.
Posted 24 days ago. May 30, 2018 - 08:38 PDT
Enhanced Logging service has returned to normal. We continue to work on restoring full functionality for Analytics and Object Store. We will have an update posted in 30 minutes.
Posted 24 days ago. May 30, 2018 - 07:58 PDT
We are also investigating issues with Enhanced logging, We will be posting an update shortly as the team is currently investigating.
Posted 24 days ago. May 30, 2018 - 05:30 PDT
We identified additional errors on API Analytics data ingest and experience increased error rate on Object Store. The engineering team is working on resolving the issues.
Posted 24 days ago. May 30, 2018 - 02:55 PDT
Insights service is back online and processing events in normal capacity. API Analytics is online but still processing queued events. Object Store continues to work normally. We are monitoring recovery.
Posted 24 days ago. May 29, 2018 - 21:08 PDT
Object Store Service has been restored. API Analytics Service has been restored, there are still some delays on processing the queued data . Insights Analytics still has high error rates.
Posted 24 days ago. May 29, 2018 - 18:56 PDT
API Analytics service has been restored and it's catching up with the data ingestion. We are still working on Insights Data ingestion. It also affected the usage of the Object Store in CloudHub, causing elevated errors on Object Store API.
Posted 24 days ago. May 29, 2018 - 16:21 PDT
We have confirmed elevated error rate with Insights and API Analytics data ingress and have identified the issue and expect to have it resolved soon.
Posted 24 days ago. May 29, 2018 - 14:40 PDT
We are experiencing elevated error rates with Insights and API Analytics data ingress. Our engineers are investigating.
Posted 24 days ago. May 29, 2018 - 14:14 PDT
This incident affected: CloudHub Services (Object Store, Insight) and API Services (Analytics).