Between September 22, 12:19 PM UTC and 01:27 PM UTC, customers experienced errors when opening diagrams without a selected view in the Next-Generation Diagrams editor.
As part of our ongoing efforts to improve the stability of our services, we wanted to address a customer-reported issue with views in the Next-Generation Diagram Editor. If a view was selected and saved in the diagram, but then removed from the view model, the editor would not work. The fix was released with a bug, despite automated and manual testing.
Our monitoring systems alerted the team immediately after the release went into production. The team rolled back the broken code, which resolved the situation, and released the improved fix the following Monday.
After mitigation, we did a thorough analysis of why our CI/CD pipeline did not detect the bug. We have already implemented additional specific test cases to cover these unforeseen scenarios.
We plan to accelerate our rollback pipeline to significantly reduce customer downtime in the Next-Generation Diagram Editor.