On December 13th, the Webhooks service faced a disruption in all regions.
Duration: [10:08 UTC] - [10:24 UTC] ~ 16 minutes
Customers were not able to see existing webhooks, nor modify them.
Event deliveries were also delayed during the disruption.
The incident was related to the Webhooks disruption on December 12th.
We rolled back the changes we made on both the days.
No data was lost during the disruption.
We introduced an improvement in the Webhooks service, which was related to authentication against endpoints. This change broke the authentication with the service, and hence all inbound calls were failing.
We have already improved testing strategy for this case and will continue to improve in the future.