Ubuntu 24.04 LTS kubelet NotReady After containerd Configuration Drift

📖 ~1 min read

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

Node becomes NotReady and workloads cannot schedule or recover.

Environment & Reproduction

Occurs after runtime upgrades or config file edits.

sudo systemctl status kubelet && kubectl get nodes

Root Cause Analysis

kubelet and containerd cgroup settings diverge, causing runtime initialization failures.

Quick Triage

Check kubelet journal and runtime endpoint status.

sudo journalctl -u kubelet --since -1h

Step-by-Step Diagnosis

Inspect containerd config and kubelet args for cgroup alignment.

sudo grep -R SystemdCgroup /etc/containerd/config.toml /var/lib/kubelet/config.yaml

Solution – Primary Fix

Enable `SystemdCgroup = true` in containerd and restart runtime and kubelet.

sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml && sudo systemctl restart containerd kubelet

Solution – Alternative Approaches

Rejoin node to cluster if local kubelet state is irrecoverably inconsistent.

Verification & Acceptance Criteria

Node returns to Ready and pods schedule successfully.

Rollback Plan

Revert runtime config changes and restart services.

Prevention & Hardening

Use configuration management to enforce consistent kubelet/runtime settings.

Related Errors & Cross-Refs

Related to CRI socket mismatch and paused image pull failures.

Related tutorial: View the step-by-step tutorial for Ubuntu 24.04 LTS.

View all Ubuntu 24.04 LTS tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

Ubuntu Kubernetes node runtime and kubelet troubleshooting docs.