Debian 9 – high CPU and disk I/O from systemd-journald

📖 ~1 min read

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

Host performance degrades due to journald write amplification and CPU pressure.

Environment & Reproduction

Triggered by noisy services, debug logging, or unconstrained journal retention.

# Capture baseline
ps -eo pid,comm,%cpu,%mem --sort=-%cpu | head
iostat -xz 1 3 2>/dev/null || true
journalctl --disk-usage

Root Cause Analysis

Excessive log volume and retention pressure increase disk writes and indexing overhead.

Quick Triage

Identify top log emitters and current journal size limits.

# Quick triage
journalctl -p warning -n 200 --no-pager
cat /etc/systemd/journald.conf | sed -n 1,120p

Step-by-Step Diagnosis

Measure write patterns and isolate service-specific log storms.

# Detailed diagnosis
journalctl -o short-iso -n 500 --no-pager | awk {print $5} | sort | uniq -c | sort -nr | head
vmstat 1 5

Solution – Primary Fix

Set sane retention caps and restart journald to reduce pressure.

# Primary fix
sudo sed -i s/^#SystemMaxUse=.*/SystemMaxUse=500M/ /etc/systemd/journald.conf
sudo sed -i s/^#RuntimeMaxUse=.*/RuntimeMaxUse=200M/ /etc/systemd/journald.conf
sudo systemctl restart systemd-journald

Solution – Alternative Approaches

Rate-limit noisy services or redirect high-volume logs to dedicated sinks.

# Alternatives
sudo journalctl --vacuum-size=300M
sudo journalctl --vacuum-time=7d

Verification & Acceptance Criteria

CPU and disk utilization normalize while required logs remain available.

# Verify
ps -eo comm,%cpu --sort=-%cpu | head
journalctl --disk-usage

Rollback Plan

Restore previous journald limits if troubleshooting visibility becomes insufficient.

# Rollback
sudo cp /etc/systemd/journald.conf.bak /etc/systemd/journald.conf 2>/dev/null || true
sudo systemctl restart systemd-journald

Prevention & Hardening

Set logging budgets and monitor per-service log rates as part of SRE hygiene.

# Hardening
journalctl --disk-usage
systemctl status systemd-journald --no-pager

Related Errors & Cross-Refs

Often coexists with rootfs pressure and noisy service crash loops.

Related tutorial: View the step-by-step tutorial for Debian 9.

View all Debian 9 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

systemd-journald configuration and log retention tuning documentation.