We are concluding this incident after confirming service stability.
Actual window of degraded service: Between 4:05 PM and 4:33 PM UTC on the 19th of March
What happened: A subset of users may have intermittently received internal server errors between 4:05 PM and 4:33 PM UTC on the 19th of March. Engineers identified the source and quickly resolved the issue. This was due to abnormal service behavior but we have adjusted monitors and resources to better handle situations like this moving forward.
Outside of the intermittent errors received from UI, we confirmed there was no impact to scheduled Rules, Data Loads, or Programs.
We apologize for any inconvenience this has caused.
Posted 3 months ago. Mar 19, 2019 - 19:49 UTC
All services are operational. We will post root cause analysis information after a detailed review.
Posted 3 months ago. Mar 19, 2019 - 17:37 UTC
We are currently getting reports of internal server timeout errors. We have isolated this issue to MDA API queries (reports, rules, etc.). We are currently working on a fix with Urgent priority and will provide updates soon.