As of 04:07 Pacific, the impact to API Manager is resolved. API Gateway is now able to retrieve and apply all the configured policies.
API Gateway might continue to show errors for the application of previous versions of the policy but this has no impact on the gateway and the errors will disappear with a restart.
Posted Mar 05, 2024 - 04:39 PST
Monitoring
After the actions taken by our technical teams, we are seeing a steady decrease in the error rate and expect to be fully out of impact within the next hour.
Posted Mar 05, 2024 - 04:01 PST
Update
We have validated a fix for this incident in our lower environments and are working to deploy it now. Once deployed and we see the error rates dropping, we will update to monitoring.
Posted Mar 04, 2024 - 23:28 PST
Update
This is a retroactive note at 7:27 PM Pacific time, we noted an inaccuracy in our original posting where we explained that existing applications without policy updates would continue to operate normally. This turned out to be incorrect. A status post was made at that time on our EU Status site, but it was not communicated here on the US Status site. Please refer to the previous message for remediation steps while we work to resolve the overall situation.
Posted Mar 04, 2024 - 21:03 PST
Identified
We have addressed the backend issue causing the errors that have generated this incident, while we work on restoring the service fully, we have identified a remediation step that can be taken to restore your applications to their expected performance.
If your Mule Gateway is currently affected, proceed to restart your Mule Runtime.
If you are unsure if your gateway is currently affected, you can review the logs to see if there is any occurrence of an error message like:
Error deploying policy to application com.mulesoft.mule.runtime.gw.policies.template.exception.PolicyTemplateAssetException: Unexpected error when requesting template metadata for
If you see occurrences of the above error message then you should proceed to restart your Mule Runtime.
Posted Mar 04, 2024 - 20:37 PST
Update
Existing applications with the policies will continue to operate smoothly. However, in the event of application restarts or when deploying new apps, there is a risk that they may encounter difficulties in downloading and applying API Manager policies, resulting in deployment failures.
Posted Mar 04, 2024 - 16:14 PST
Update
We are continuing to investigate this issue.
Posted Mar 04, 2024 - 16:06 PST
Investigating
Customers may experience errors while using applying new policies to existing and new applications. Our teams are investigating.
Posted Mar 04, 2024 - 14:56 PST
This incident affected: Anypoint Management Center (API Manager).