Debian 11: /var fills up due to systemd journal growth

📖 ~1 min read

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

Disk usage in `/var` spikes until services fail from low-space conditions.

Environment & Reproduction

Applications report no space left on device and journald directories consume most storage.

Root Cause Analysis

Run `df -h /var`, `du -sh /var/log/journal/*`, and `journalctl –disk-usage` to quantify impact.

Quick Triage

Unlimited or oversized journal retention combined with noisy services generating excessive logs.

Step-by-Step Diagnosis

Set `SystemMaxUse` and `RuntimeMaxUse` in journald config, then vacuum with `sudo journalctl –vacuum-time=14d`.

Solution – Primary Fix

Check freed capacity using `df -h /var` and validate new journal limits are enforced after restart.

Solution – Alternative Approaches

Define log retention policy by host class and cap verbose debug logging in production.

Verification & Acceptance Criteria

Restore prior journald configuration if reduced retention conflicts with audit requirements.

Rollback Plan

Alert on `/var` usage trend and journal growth rate to prevent emergency cleanups.

Prevention & Hardening

`journalctl –disk-usage`; `sudo journalctl –vacuum-time=14d`; `df -h /var`

Related Errors & Cross-Refs

Include top emitting services and recent high-volume log excerpts for root-cause review.

Related tutorial: View the step-by-step tutorial for debian-11.

View all debian-11 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

Log forwarding to centralized systems can reduce local retention pressure on small disks.