Service Disruption for european workspaces
Incident Report for LeanIX
Postmortem

On 26th of September 7:21 UTC our Engineering teams were alerted that our customer instance eu.leanix.net reached the limit of its usable database connections. This has been indicated by our first status update of that day.

At the same time the teams identified that a bug, introduced through a release, has been the root cause of it. This bug partly affected other customer instances such as au.leanix.net, so that some customers could not login anymore. At 7:31 UTC this change was rolled back and all instances except eu.leanix.net were fully operational again.

As the release happened parallel to several larger customer integrations on eu.leanix.net it took until 08:54 UTC until the pending database connections were freed up again and our teams began monitoring the mitigated state. To prevent further performance issues we increased the available memory on the affected components on eu.leanix.net.

No data was lost. We sincerely apologize for any inconvenience this incident may have caused and appreciate your patience.

Posted Sep 27, 2022 - 08:35 CEST

Resolved
This incident has been resolved.
Posted Sep 26, 2022 - 14:18 CEST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Sep 26, 2022 - 10:54 CEST
This incident affected: EU Instances (EAM).