We've resolved the issue now. Our primary database failed, and to prevent data inconsistencies the cluster didn't automatically fail over to another server. We manually failed over, then had to complete some more work to bring the other servers back online with the new master. It did mean that from the point where we brought the master back online (approx 5:17), it was operating on its own until a few minutes ago which meant service was very, very slow.
Sorry for the disruption this afternoon.
Posted Jun 04, 2014 - 18:14 BST
Update
We're trying to restore the service level now, we're running slowly.
Posted Jun 04, 2014 - 17:38 BST
Update
Service is fully restored, data will be fully in order. We'll update with more details here shortly.
Posted Jun 04, 2014 - 17:17 BST
Update
You may see cached/older data until we resolve this issue.
Posted Jun 04, 2014 - 17:12 BST
Identified
We have problem with our database cluster, which we're looking at now. More information to follow.
Posted Jun 04, 2014 - 17:03 BST
Investigating
We're aware of problems accessing the service at the moment, and we're investigating now. We'll update with more information here in the next 30 minutes. Thanks for your patience.