On June 11th 2024 between 4:35 UTC and 6:20 UTC, LeanIX Pathfinder API was inaccessible for customer workspaces.
During the given timeframe, customers were not able to access workspaces. Due to a fault in our release pipeline, broken code was deployed to production, which resulted in an overall downtime of two hours for all customers and workspaces. A rollback was not possible, and a full release of the affected service was required.
Our team identified the issue and needed to prepare a full release to fix the broken code. Service availability was established by 6:20 UTC.
After the incident, we took steps to understand why the release was not stopped by our CI/CD pipeline checks and how we could prevent similar incidents in the future. We have identified several areas for improvement and are currently working on: