RTF Clusters failing to forward logs into Anypoint Monitoring
Incident Report for US1 - Anypoint Platform
Resolved
The team has resolved the logging issue for Runtime Fabric. The fix is released for all Supported Mule Runtime versions.
MuleSoft highly recommends updating their RTF runtimes to the latest release. Please refer to the Knowledge Article for further information. https://help.mulesoft.com/s/article/Logs-of-Mule-applications-running-in-Runtime-Fabric-are-not-visible-in-Anypoint-Monitoring-during-the-last-week-of-December-2021. An RCA will be available in 5 business days.
Posted Dec 27, 2021 - 23:20 PST
Update
A fix has been released for 4.4.0, 4.3.0, 4.2.2, 4.2.1, and 3.9.5 versions of runtimes. Therefore, we recommend that customers upgrade their applications in the above runtime versions to the latest release to resolve this issue. The team continues to work on releasing the fix for all other supported runtime versions, and the status page will be updated once the fix is available. The next update will be in approximately 2 hours.
Posted Dec 27, 2021 - 22:51 PST
Update
The team continues to work to implement the fix. The next update will be in approximately 2 hours.
Posted Dec 27, 2021 - 21:30 PST
Update
The team has identified the root cause and continues to work on the fix. The next update will be in approximately 2 hours.
Posted Dec 27, 2021 - 19:26 PST
Identified
The issue has been identified and Engineers are currently working on a fix.
Posted Dec 27, 2021 - 18:22 PST
Update
Engineers are putting together a query to help isolate the cause, in efforts of mitigating the issue.
Posted Dec 27, 2021 - 17:10 PST
Update
Engineers are still investigating the trigger/cause of the issue and are currently in a trouble-shooting call session.
Posted Dec 27, 2021 - 15:45 PST
Update
As Engineers continue to investigate, we have been able to establish/verify that the logs are being generated and available for search.
Posted Dec 27, 2021 - 14:36 PST
Update
Engineering teams continue to investigate in an effort to isolate the cause/trigger of the issue, we are walking through the data-flow to identify where we might have a potential gap(s) that might be affecting the log-pulls to the UI.
Posted Dec 27, 2021 - 13:41 PST
Investigating
MuleSoft is aware of a problem on Runtime Fabric limited to log retrieval as apps are working as expected, Engineering teams are currently investigating the issue.
Posted Dec 27, 2021 - 12:36 PST
This incident affected: Runtime Services (Runtime Fabric).