Database outage

Incident Report for Oliver Booth

Postmortem

The incident was caused by a duplicate MariaDB server being installed when attempting to install Froxlor for multi-site management. Froxlor has since been uninstalled and MariaDB was reconstructed in a freshly pulled docker image and a configuration backup.

Let this be a lesson to those who test on prod.

Posted Oct 16, 2024 - 16:44 BST

Resolved

The database is operational and service has been resumed.

Posted Oct 16, 2024 - 16:42 BST

Monitoring

Configuration has been restored from a backup and the MariaDB container has been regenerated. I'm currently monitoring for issues.

Posted Oct 16, 2024 - 16:41 BST

Identified

The issue has been identified as corrupted configuration files caused by a duplicate installation of the MariaDB server.

Posted Oct 16, 2024 - 16:40 BST

Investigating

I'm aware of an outage affecting the ability to connect to the database.

Posted Oct 16, 2024 - 16:39 BST

This incident affected: oliverbooth.dev (Database).