Database won't start: MySQL, MariaDB and PostgreSQL recovery

Updated November 2026

When a database refuses to start, the error log usually names the reason: a full disk, wrong file ownership, a bad setting or a crash that needs recovery. Read the log first, copy the data directory before you try anything risky, and treat forced recovery as a way to get a dump out, not as a fix.

Symptoms

• systemctl start mariadb (or mysql, mysqld, postgresql) fails or the service starts and dies after a few seconds.
• Web pages show “Error establishing a database connection” or a 500/502 error.
• Log lines such as “No space left on device”, “Permission denied”, “InnoDB: Database page corruption”, or PostgreSQL “PANIC: could not write to file pg_wal/…”.

Step 1: read the error log

# MariaDB (Debian/Ubuntu and RHEL family)
sudo journalctl -u mariadb -n 100 --no-pager
sudo tail -n 100 /var/log/mysql/error.log        # Debian/Ubuntu, if file logging is on
sudo tail -n 100 /var/log/mariadb/mariadb.log    # RHEL family

# MySQL
sudo journalctl -u mysql -n 100 --no-pager       # Debian/Ubuntu (unit: mysql)
sudo tail -n 100 /var/log/mysql/error.log        # Debian/Ubuntu
sudo tail -n 100 /var/log/mysql/mysqld.log       # RHEL family (unit: mysqld)

# PostgreSQL
sudo journalctl -u 'postgresql*' -n 100 --no-pager
sudo ls /var/log/postgresql/                     # Debian/Ubuntu: postgresql-17-main.log etc.
sudo ls /var/lib/pgsql/data/log/                 # RHEL family

Read from the first error, not the last line. The last line is often just “aborting”.

Step 2: disk full

df -h /var/lib /var/log /tmp
df -i /var/lib
sudo du -xh --max-depth=1 /var/lib/mysql /var/lib/postgresql 2>/dev/null | sort -h | tail

Free space from other things first: old logs, package caches, stray backups. See Disk full. Do not delete database files by hand.

• MySQL/MariaDB binary logs (binlog.000123, mysql-bin.000123) can be large. Do not delete them with rm: once the server is up, remove old ones with PURGE BINARY LOGS BEFORE '2026-10-27 00:00:00';, after making sure no replica still needs them.
• PostgreSQL WAL. The PostgreSQL docs warn that if the disk holding the WAL files (pg_wal) fills up, the server can panic and shut down. Never delete files from pg_wal: you can make the database unrecoverable. Free space elsewhere, start the server, then find why WAL piled up (step 5).

Step 3: permissions and configuration

After a restore, a copy or a disk move, files often belong to the wrong user.

sudo ls -ld /var/lib/mysql /var/lib/mysql/*  | head
sudo ls -ld /var/lib/postgresql/*/main      # Debian/Ubuntu
sudo ls -ld /var/lib/pgsql/data             # RHEL family

# Fix ownership only if it is wrong
sudo chown -R mysql:mysql /var/lib/mysql
sudo chown -R postgres:postgres /var/lib/postgresql   # or /var/lib/pgsql
sudo chmod 700 /var/lib/postgresql/17/main            # PostgreSQL requires 0700 (or 0750)

On RHEL with SELinux, a data directory in a new location needs the right label. Run sudo ausearch -m avc -ts recent to see denials, and sudo restorecon -Rv /var/lib/mysql for the default path. On Ubuntu, AppArmor can block a moved MySQL data directory in the same way. If the log names a bad setting after an upgrade, comment it out in the config file and try again.

Step 4: copy before you repair

sudo systemctl stop mariadb          # or mysql / mysqld / postgresql
sudo rsync -a /var/lib/mysql/ /mnt/backup/mysql-copy-$(date +%F)/

Use a different disk if you can. Every step below can make things worse, and this copy lets you try again. If the data is valuable and the log shows hardware errors (I/O errors in journalctl -k), stop here and get professional help.

MySQL and MariaDB: InnoDB crash recovery

After a power cut or crash, InnoDB replays its redo log automatically. Give it time: on a large server this can take many minutes. Only if it keeps failing with corruption errors should you consider innodb_force_recovery.

Procedure, one step at a time:

1. Add to the [mysqld] section (MariaDB also reads [mariadb]) of the server config, for example /etc/mysql/mariadb.conf.d/50-server.cnf, /etc/mysql/mysql.conf.d/mysqld.cnf or /etc/my.cnf.d/:

   [mysqld]
   innodb_force_recovery = 1

2. Start the server. If it still fails, raise the value by one and try again. Stop at 3 if you can.
3. When it starts, dump everything at once. MySQL blocks writes at any value above 0; MariaDB allows some writes at 3 or below, but do not write anyway.

   # MariaDB
   sudo mariadb-dump --all-databases --routines --events --single-transaction > /mnt/backup/all.sql
   # MySQL
   sudo mysqldump --all-databases --routines --events --single-transaction > /mnt/backup/all.sql

   If one table makes the dump crash, dump the other databases one by one and deal with that table separately.
4. Remove the innodb_force_recovery line. Never leave it set.
5. Build a clean server: stop it, move the old data directory aside (do not delete it), initialize a new one (mariadb-install-db --user=mysql for MariaDB, mysqld --initialize --user=mysql for MySQL), start it and import the dump with sudo mariadb < all.sql. For MySQL, --initialize writes a temporary root password to the error log: log in with mysql -u root -p, set a new password with ALTER USER, then import with mysql -u root -p < all.sql.
6. Check the result: sudo mariadb-check --all-databases or sudo mysqlcheck --all-databases.

PostgreSQL: WAL and crash recovery

PostgreSQL also recovers automatically by replaying WAL (write-ahead log) at start-up. Most “won’t start” cases are a full disk, permissions, or a config error, not corruption. Check the cluster state, then the reason WAL grew:

sudo -u postgres /usr/lib/postgresql/17/bin/pg_controldata /var/lib/postgresql/17/main | grep state
# RHEL family: sudo -u postgres pg_controldata /var/lib/pgsql/data

Once it runs, find what holds WAL back. The PostgreSQL docs list a failing archive_command or archive_library, a slow or dead standby with a replication slot, and a large wal_keep_size:

sudo -u postgres psql -c "SELECT slot_name, active, restart_lsn FROM pg_replication_slots;"
sudo -u postgres psql -c "SELECT failed_count, last_failed_wal, last_failed_time FROM pg_stat_archiver;"
# Drop a slot only if you are sure its standby is gone for good:
# sudo -u postgres psql -c "SELECT pg_drop_replication_slot('old_standby');"

Restore from a dump instead

If you have a recent good backup, restoring it is often faster than repair and gives you known-good data:

# MySQL/MariaDB (plain SQL dump)
sudo mariadb < /backup/all-2026-11-02.sql          # or: sudo mysql < ...
# PostgreSQL plain SQL from pg_dumpall
sudo -u postgres psql -f /backup/all-2026-11-02.sql postgres
# PostgreSQL custom-format dump of one database
sudo -u postgres createdb shop
sudo -u postgres pg_restore -d shop /backup/shop-2026-11-02.dump

Then replay what you can from binary logs or WAL archives if you use point-in-time recovery. See Restore from backup.

Confirm it is fixed

• The service stays up after a restart, with no errors in the log.
• Row counts of key tables match what you expect.
• mariadb-check / mysqlcheck report OK, or PostgreSQL queries on the main tables succeed.
• The application works and a fresh backup has been taken.

Prevent it next time

• Alert on disk usage at 80% and keep the database on its own filesystem.
• Take nightly dumps and test a restore every month. See Ransomware-proof backups.
• Monitor replication slots and WAL archiving in PostgreSQL, and binary log size in MySQL/MariaDB.
• Use reliable storage, and do not turn off durability settings such as PostgreSQL’s fsync to gain speed.

When to get help

Get a database specialist or data recovery service if the log shows disk I/O errors, if you would need innodb_force_recovery above 3 or pg_resetwal on data you cannot lose, or if no backup exists. Work only on copies from then on.

Sources

• Forcing InnoDB Recovery, MySQL 8.4 Reference Manual
• InnoDB Recovery Modes, MariaDB Documentation
• pg_resetwal, PostgreSQL Documentation
• Monitoring Disk Usage (Disk Full Failure), PostgreSQL Documentation
• WAL Configuration, PostgreSQL Documentation