Incident Description

Following a deployment on 2024-10-14 at 14:09 UTC, all customers across all regions experienced an unresolved (infinite) loading indicator when attempting to access reports and diagrams from the bookmarks overview page.

No customer data was affected.

The service disruption lasted for approximately 40 minutes until 2024-10-14 at 14:49 UTC.

Incident Resolution

Reverting the code change that caused the incident resolved the issue.

Root Cause Analysis

A minor code change intended to extend our usage monitoring introduced a bug, which led to the infinite loading page, preventing the user to see any Reports or Diagrams.

Preventative Measures

We have identified the combination of factors that led to this oversight and will enforce the following practices to prevent similar issues in the future:

1. Add stricter rules in our CI/CD to prevent such code change of happening again.
2. Improve the current test scenarios during the deployment of new releases.
3. Enforce more thorough local and automated testing of all code changes.