We are sharing planned and unplanned service disruptions of the Tideways Performance Monitoring and Profiling service here. If you are experiencing problems or want to inquire about ongoing problems please write to email@example.com.
We have got a notification that our Elasticserach cluster has problems. Processing of data is delayed.
Edit 22:10: The master node of the Elasticsearch cluster went offline, a new master was elected and data is started to get synchronized. We saw 2-3 minutes of slow performance due to workers and frontend still trying to access very slow node before it went offline. Processing is back to normal speeds now and queue is catched up.
Edit 22:15 Node 3 of the ES cluster is back online and rebalancing has again picked up. Processing performance is back to normal.
Edit 22:34 Rebalancing after the node got back into the cluster is finished.