Final Update: Tuesday, 03 August 2021 04:37 UTC
We've confirmed that all systems are back to normal with no customer impact as of 08/03, 04:36 UTC. Our logs show the incident started on 08/02, 19:48 UTC and that during the 8 Hours & 48 minutes hours that it took to resolve the issue. Some customers may have experienced query failures, delayed or missed Log Search Alerts, data latency, delayed data ingestion and incorrect alert activation in WestUS2 region.
- Root Cause: The failure was due to an issue with one of the dependent services.
- Incident Timeline: 8 Hours & 48 minutes - 08/02, 19:48 UTC through 08/03, 04:36 UTC
We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.
-Vyom
Update: Tuesday, 03 August 2021 02:11 UTC
Root cause has been isolated to Storage issues in WUS2 region starting 19:48 UTC Aug 2, 2021 which was impacting Ingestion under Azure log analytics area. To address this issue mitigation steps were applied. Currently Ingestion is now working as expected. Some customers may experience data latency until all the messages in the backlog queues for ingestion are cleared and we estimate 2 hours before all ingestion data backlog is addressed.
- Work Around: none
- Customer Impact: Customers may see delayed data ingestion and incorrect alert activation for the duration of this incident
- Next Update: Before 08/03 04:30 UTC
-Arish B
Update: Monday, 02 August 2021 23:54 UTC
Root cause has been isolated to Storage issues in WUS2 region starting 19:48 UTC Aug 2, 2021 which was impacting Ingestion under Azure log analytics area. To address this issue mitigation steps were applied. Currently Ingestion is now working as expected. Some customers may experience data latency until all the messages in the backlog queues for ingestion are cleared and we estimate 2 hours before all ingestion data backlog is addressed.
- Work Around: none
- Customer Impact: Customers may see delayed data ingestion and incorrect alert activation for the duration of this incident
- Next Update: Before 08/03 02:00 UTC
-Arish B
Update: Monday, 02 August 2021 21:31 UTC
We are aware of issues within Azure Log analytics and Root cause has been identified with one of the services is facing storage issues and engineers are working on applying mitigations. Some customers may experience delayed or missed Log Search Alerts and Data Latency in WUS2 region starting 19:48 UTC Aug 2, 2021
- Work Around:
- Customer Impact: Customers may see delayed data ingestion and incorrect alert activation for the duration of this incident
- Next Update: Before 08/02 23:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Arish B
