Thursday 23rd July 2015

Degrading Performance

After a deployment CPU spiked on one of our backend servers, leaving both Collector and UI with degraded performance.

Our backlog of tasks doubled.

We have identified the problem to be related to a configuration change in php-fpm and reverted the changes. The performance is slowly stabilizing now, but the backlog has to be processed and causes additional load that affects the system.