High Swap Usage and OOM Killer Events

📖 ~1 min read

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

Server becomes slow, processes are killed, and critical services restart due to memory exhaustion and swap thrashing.

Environment & Reproduction

Typical on small Ubuntu 16.04 instances running memory-heavy workloads without tuned limits. Reproduce by overcommitting RAM-intensive jobs.

Root Cause Analysis

Memory demand exceeds physical RAM and swap behavior causes heavy I/O contention, eventually triggering Linux OOM killer decisions.

Quick Triage

Use free -m, vmstat 1, and top to confirm pressure. Check dmesg and journalctl -k for oom-killer victim logs.

Step-by-Step Diagnosis

Identify top memory consumers over time with ps and smem, review cgroup limits, and inspect swappiness and overcommit sysctl settings.

Solution – Primary Fix

Right-size workload or host RAM, reduce memory leaks, tune vm.swappiness, and set service-level memory limits/restart policy under systemd.

Solution – Alternative Approaches

Add or resize swap cautiously, distribute workload horizontally, or isolate memory-hungry components to dedicated nodes.

Verification & Acceptance Criteria

No new OOM events for agreed observation window, swap usage stabilizes, and latency returns to baseline.

Rollback Plan

Revert sysctl changes and service memory limits, then restore prior deployment sizing if regression is observed.

Prevention & Hardening

Define memory SLOs, set alerting on pressure metrics, and enforce load tests that include worst-case memory patterns.

Related Errors & Cross-Refs

Can manifest alongside MySQL crashes, Java GC pauses, and kernel soft lockups under heavy swap I/O.

Related tutorial: View the step-by-step tutorial for Ubuntu 16.04 LTS.

View all Ubuntu 16.04 LTS tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

proc(5), sysctl tuning references, and capacity planning guides for Linux memory management.