Issue: Application Availability Impact
Cause:
An infrastructure bug led to system instability leaving a small number of API nodes accepting traffic at the time.
Timeline on the 12th of November:
9:24 UTC - Synthetic monitoring alerts and customer reports signaled API issues.
9:31 UTC - Engineers confirmed the issue and acted quickly to bring supporting systems online.
9:43 UTC - The incident status was closed after observing a period of stability.
Resolution:
Engineers quickly started the recovery process by introducing new nodes to balance API load and retired the affected systems.
We have also introduced additional checks to avoid failures like this moving forward.
Please email support@gainsight.com if there are further questions.