FreeBSD 13 Swap Exhaustion Caused by Unbounded ZFS ARC Growth

📖 ~1 min read

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

System enters heavy swap and latency spikes across all workloads.

Environment & Reproduction

Seen on mixed database/file hosts with no ARC cap policy.

swapinfo -h && top -aSH

Root Cause Analysis

ARC competes with application memory until paging and thrash begin.

Quick Triage

Capture ARC size and page daemon activity.

sysctl kstat.zfs.misc.arcstats.size vm.stats.vm.v_swappgsout

Step-by-Step Diagnosis

Correlate memory consumers and ARC reclaim behavior.

arc_summary | head -120

Solution – Primary Fix

Set vfs.zfs.arc_max to a tested ceiling and persist in loader config.

sudo sysctl vfs.zfs.arc_max=8589934592

Solution – Alternative Approaches

Split storage and compute roles or add RAM/swap capacity.

Verification & Acceptance Criteria

Swap churn drops and steady-state latency returns to baseline.

Rollback Plan

Revert ARC cap and restart during controlled maintenance window.

Prevention & Hardening

Establish memory budgets per workload class with ARC policy guardrails.

Related Errors & Cross-Refs

Related to OOM kills in jailed workloads and stalled I/O queues.

Related tutorial: View the step-by-step tutorial for freebsd-13.

View all freebsd-13 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

OpenZFS ARC sizing and FreeBSD VM behavior documentation.