Service Disruption in Reports
Incident Report for SAP LeanIX
Postmortem

Summary

On June 4th 2024 at 13:33 UTC, reports were not loaded for various customer workspaces.

What happened?

In line with our continuous efforts to enhance the performance of our reporting system, we identified a key area for improvement within the report loading functionality. We have optimized this service to load specific data segments associated with the workspace in parallel.

The side-effect of this change caused several requests to be in waiting state during a steady load on the system.

Mitigation: What did we do about it?

We’ve identified the issue after monitoring the release, and reverted the change into production. Leading to a healthy state 30 minutes after the incident started, at 14:00 UTC.

Follow-ups: How will we improve?

After mitigation, we identified the issue introduced with the new change and we are putting our efforts to fix it. We’ll also extend the coverage of our QA process to also include performing load tests ensuring better reliability.

Posted Jun 06, 2024 - 13:09 UTC

Resolved
This incident has been resolved.
Posted Jun 04, 2024 - 14:30 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jun 04, 2024 - 14:03 UTC
Identified
We are currently experiencing a service disruption in the Reports area. Reports might not load as expected. Our team has identified the root cause and is working on a fix.

We will send an additional update in 30 minutes.
Posted Jun 04, 2024 - 13:54 UTC
This incident affected: EU Instances (EAM), US Instances (EAM), CA Instances (EAM), AU Instances (EAM), DE Instances (EAM), CH Instances (EAM), AE Instances (EAM), UK Instances (EAM), and BR Instances (EAM).