CS - EU - Investigating Delays
Incident Report for Gainsight
Resolved
Incident:
Some EU customers experienced delays in Rules and Journey Orchestrator on the 8th of June, 2023.

Root Cause:
After noticing the delays, Engineers temporarily paused related queues to investigate.
While troubleshooting, we discovered network connectivity issues tied to underlying hardware.

Recovery Action :
While still in a paused state, action was taken to assign affected resources to new hardware.
Once back online with new hardware, jobs were processing as expected.

Preventive Measures:
Redundant architecture design and quick action helped prevent major impact. Still, we are inspecting monitoring, alerting, and automation to ensure optimal design.
Additionally, we are confirming self-healing configurations with upstream service providers.
Posted Jun 08, 2023 - 17:00 UTC