Incident:
Beginning around 12:00 UTC on the 13th of June, Engineers were alerted of elevated queue levels for Connector services in CS-US1.
Root Cause:
A leader node was found to have higher than usual disk activity which prevented optimal job execution for Connector services.
Recovery Action:
Engineers scaled the number of Connector instances to correct the issue temporarily. Additionally, Engineers skipped long-running and duplicate jobs to help recover.
Preventive Measures:
System configuration adjustments have been made to prevent these issues moving forward.