Debian 9 – software RAID array stays degraded and rebuild does not start

📖 ~1 min read

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

RAID redundancy is lost and data risk increases while rebuild remains stalled.

Environment & Reproduction

Occurs after member disk failure, metadata mismatch, or replacement workflow issues.

# Capture baseline
cat /proc/mdstat
mdadm --detail /dev/md0
lsblk -f

Root Cause Analysis

Array cannot resync due to failed members, missing superblock compatibility, or blocked rebuild state.

Quick Triage

Validate failed disk state and detect whether replacement is recognized.

# Quick triage
mdadm --examine /dev/ | head -n 80
dmesg | grep -Ei md|raid|error | tail -n 120

Step-by-Step Diagnosis

Inspect md metadata and kernel logs to determine why rebuild is not advancing.

# Detailed diagnosis
cat /proc/mdstat
mdadm --detail /dev/md0
udevadm info --query=all --name=/dev/ | head -n 80

Solution – Primary Fix

Fail and remove bad member, add validated replacement, and monitor rebuild progress.

# Primary fix
sudo mdadm /dev/md0 --fail /dev/ --remove /dev/
sudo mdadm /dev/md0 --add /dev/
watch -n 5 cat /proc/mdstat

Solution – Alternative Approaches

Recreate array from backup metadata only if direct rebuild cannot proceed safely.

# Alternatives
sudo mdadm --assemble --scan
mdadm --examine --scan

Verification & Acceptance Criteria

Array returns to clean state and sync completes with no failed members.

# Verify
cat /proc/mdstat
mdadm --detail /dev/md0

Rollback Plan

Restore from backup or standby storage if array health remains unstable.

# Rollback
mdadm --stop /dev/md0
mdadm --assemble --scan

Prevention & Hardening

Monitor SMART and md events and maintain tested disk replacement runbooks.

# Hardening
cat /etc/mdadm/mdadm.conf
smartctl -H /dev/ 2>/dev/null || true

Related Errors & Cross-Refs

Related to mdadm cannot add disk, superblock mismatch, and degraded array warnings.

Related tutorial: View the step-by-step tutorial for Debian 9.

View all Debian 9 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

mdadm administration and Linux software RAID recovery documentation.