πŸ“– ~1 min read

Table of contents
  1. Symptom & Impact
  2. Environment & Reproduction
  3. Root Cause Analysis
  4. Quick Triage
  5. Step-by-Step Diagnosis
  6. Solution – Primary Fix
  7. Solution – Alternative Approaches
  8. Verification & Acceptance Criteria
  9. Rollback Plan
  10. Prevention & Hardening
  11. Related Errors & Cross-Refs
  12. References & Further Reading

Symptom & Impact

mysql service stays failed, causing API errors and application downtime. Database-dependent services cannot initialize.

Environment & Reproduction

Common on Ubuntu 16.04 with MySQL/MariaDB after abrupt power loss, disk full events, or forced restarts during writes.

Root Cause Analysis

InnoDB crash recovery cannot complete due to corrupted redo logs, permission issues, or insufficient disk space.

Quick Triage

Check systemctl status mysql and tail /var/log/mysql/error.log. Confirm free space and ownership of /var/lib/mysql.

Step-by-Step Diagnosis

Review the exact startup failure stage, test innodb_force_recovery progression carefully, and validate filesystem integrity before write operations.

Illustrative mockup for ubuntu-16-04-lts β€” ubuntu1604-b01-p18-diagnosis
mysqld error log indicates InnoDB recovery issue β€” Illustrative mockup β€” Progressive Robot

Solution – Primary Fix

Resolve space/permission issues, run controlled recovery, back up data immediately, then restore normal InnoDB settings and restart mysql under systemd.

Still having issues? Our IT Solutions & Services team can diagnose and resolve this for you. Get in touch for a free consultation.

Illustrative mockup for ubuntu-16-04-lts β€” ubuntu1604-b01-p18-fix
recovery sequence completed and mysql online β€” Illustrative mockup β€” Progressive Robot

Solution – Alternative Approaches

Restore from latest consistent backup and replay binlogs, or fail over to replica if recovery risk is high.

Verification & Acceptance Criteria

mysql is active, replication/app connections are healthy, and integrity checks on critical tables pass.

Rollback Plan

Revert to pre-recovery snapshot or backup restore path if data inconsistency appears after restart.

Prevention & Hardening

Enforce graceful shutdown, monitor disk headroom, validate backups regularly, and use replication for fast recovery.

Closely related to /var full incidents, OOM events, and filesystem corruption alerts.

Related tutorial: View the step-by-step tutorial for Ubuntu 16.04 LTS.

View all Ubuntu 16.04 LTS tutorials on the Tutorials Hub β†’

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

MySQL crash recovery docs, InnoDB internals references, and database incident response runbooks.

Need Expert Help?

If you cannot resolve this yourself, our team offers hands-on Server Management, Managed IT Services, and flexible Support Plans. Contact us today β€” we respond within one business day.