IBM AIX 7.3 – Common Problem 152 – Diagnosis and Fix

📖 ~1 min read

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

Services that log to /var begin to fail, errdemon stops writing to errlog, and syslog rotation halts. Critical diagnostic data is lost during the outage window.

Environment & Reproduction

Common on long-running LPARs where /var/adm/ras and /var/spool grow without monitoring.

df -g /var
oslevel -s
uname -a

Root Cause Analysis

Root cause is exhaustion of free blocks or inodes in the JFS2 /var filesystem, typically from large errlog, IBM.* RSCT trace, or printer spool files.

Quick Triage

Check usage of /var, identify top consumers, and confirm whether the filesystem can be grown online.

df -g /var

Step-by-Step Diagnosis

Capture deeper evidence to isolate the failure path.

df -g /var
du -sm /var/* 2>/dev/null | sort -rn | head
lsfs /var
ls -lhS /var/adm/ras | head

Solution – Primary Fix

Apply the proven primary fix in a controlled change window.

chfs -a size=+1G /var
errclear 30
find /var/adm/ras -name '*.old' -mtime +14 -delete
stopsrc -s syslogd; startsrc -s syslogd

Solution – Alternative Approaches

Use these alternatives when the primary fix is blocked by environmental constraints.

rmlv -f oldlogslv
mklv -y newvarlv -t jfs2 rootvg 4
backup -f /tmp/var.bk /var

Verification & Acceptance Criteria

Confirm the system meets acceptance criteria after the change.

df -g /var
errpt | head
lssrc -s syslogd
lssrc -s errdemon

Rollback Plan

Revert cleanly if regressions appear during validation.

chfs -a size=-1G /var
lssrc -s syslogd
errdemon -l

Prevention & Hardening

Reduce recurrence with monitoring and preventive tuning.

errdemon -B 16384
crontab -l | grep errclear
topas -P

Related Errors & Cross-Refs

Related symptoms include srcmstr failures, ODM updates rolling back, and login delays due to lastlog writes failing.

Related tutorial: View the step-by-step tutorial for aix-7.3.

View all aix-7.3 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

IBM AIX 7.3 Performance Management, errdemon and chfs man pages, IBM Support technote on /var growth.