RHEL 9: journalctl Logs Consuming Excessive Disk Space

📖 ~1 min read

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

Root filesystem usage grows quickly due to persistent systemd journal files.

Environment & Reproduction

Run `journalctl –disk-usage` and inspect `/var/log/journal` consumption.

Root Cause Analysis

Confirm `systemd` package integrity and expected journald behavior on current release.

Quick Triage

Check `systemctl status systemd-journald` and current runtime configuration.

Step-by-Step Diagnosis

Identify noisy units with `journalctl -p warning..alert –since -24h` and service-specific views.

Solution – Primary Fix

Vacuum and cap logs using `journalctl –vacuum-time=7d` and limits in `/etc/systemd/journald.conf`.

Solution – Alternative Approaches

Firewall is not root cause, but check network-facing daemons for repeated error storms.

Verification & Acceptance Criteria

Review AVC floods that may spam logs and address policy mismatches rather than muting alerts.

Rollback Plan

Restart journald and verify retention settings persist after reboot.

Prevention & Hardening

Monitor high-volume services and implement log rate controls where appropriate.

Related Errors & Cross-Refs

If retention limits are too strict, restore previous journald settings from backup.

Related tutorial: View the step-by-step tutorial for rhel-9.

View all rhel-9 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

Use `man journald.conf` and `man journalctl` for retention and vacuum controls.