Final Update: Tuesday, 07 December 2021 02:03 UTC
We've confirmed that all systems are back to normal with no customer impact as of 12/7, 02:00 UTC. Our logs show the incident started on 12/7, 00:01 UTC and that during the two hours that it took to resolve the issue 20% of customers experienced latency in querying metrics for app services and azure functions, resulting in empty metrics charts and graphs in the Azure Portal.
- Root Cause: A recent tenant upgrade caused a surge of retries which overloaded a back-end system. This was mitigated by temporarily throttling the query retries, which allowed the backend component to recover.
- Incident Timeline: 2 Hours - 12/7, 00:01 UTC through 12/7, 02:00 UTC
We understand that customers rely on Azure Monitor Metrics as a critical service and apologize for any impact this incident caused.
-Jack Cantwell
