Amazon Linux 2 — kernel-livepatch-5.10.186-179.751 — multiple vulnerabilities (9 CVEs) — patch and remediation guide

📖 ~4 min read  •  Source: Amazon Linux advisory ALAS2LIVEPATCH-2023-155

Related CVEs: CVE-2023-3777 CVE-2023-4004 CVE-2023-4147 CVE-2023-4206 CVE-2023-4207 CVE-2023-4208 CVE-2023-4622 CVE-2023-4623  +1 more

Upstream summary: A use-after-free vulnerability in the Linux kernel's netfilter: nf_tables component can be exploited to achieve local privilege escalation. When nf_tables_delrule() is flushing table rules, it is not checked whether the chain is bound and the chain's owner rule can also release the objects in certain circumstances. We recommend upgrading past commit 6eaf41e87a223ae6f8e7a28d6e78384ad7e407f8. (CVE-2023-3777) A use-after-free flaw was found in the Linux kernel's netfilter in the

Table of contents

1. Symptom & Impact
2. Environment & Reproduction
3. Root Cause Analysis
4. Quick Triage
5. Step-by-Step Diagnosis
6. Solution – Primary Fix
7. Solution – Alternative Approaches
8. Verification & Acceptance Criteria
9. Rollback Plan
10. Prevention & Hardening
11. Related Errors & Cross-Refs
12. References & Further Reading

Symptom & Impact

On Amazon Linux 2 hosts that have kernel-livepatch-5.10.186-179.751 installed, operators report behaviour consistent with Amazon Linux advisory ALAS2LIVEPATCH-2023-155: yum refuses to install or restart affected services, SELinux AVC denials appear in /var/log/audit/audit.log, and — for security-rated advisories — the host is exposed to the vulnerability set above. Impact ranges from a single service-restart loop on a single EC2 instance to wider availability incidents whenever kernel-livepatch-5.10.186-179.751 sits on the serving path of an Auto Scaling group or ECS task.

Environment & Reproduction

Reproduction targets Amazon Linux 2. Confirm release and the installed package:

cat /etc/system-release
cat /etc/os-release
rpm -q kernel-livepatch-5.10.186-179.751
yum info kernel-livepatch-5.10.186-179.751 | head -20

Trigger the workflow that exposes kernel-livepatch-5.10.186-179.751 — multiple vulnerabilities (9 CVEs) — patch and remediation guide while collecting:

sudo journalctl -u kernel-livepatch-5.10.186-179.751 -b --no-pager | tail -200
sudo journalctl -xe --no-pager | tail -200
sudo tail -200 /var/log/yum.log
sudo tail -200 /var/log/audit/audit.log
# For an evidence bundle bundle with sosreport (Amazon Linux ships it):
sudo sosreport --batch

For fleet-wide visibility, query Amazon Inspector and SSM at the same time:

aws inspector2 list-findings 
  --filter-criteria 'awsAccountId={comparison=EQUALS,value=<account-id>}' 
  --max-results 50
aws ssm describe-instance-patches --instance-id <i-xxxx> | head -40

Root Cause Analysis

Root cause is documented in Amazon Linux advisory ALAS2LIVEPATCH-2023-155. The Amazon Linux Security Team shipped fixes in the corresponding kernel-livepatch-5.10.186-179.751 update for Amazon Linux 2; running an outdated AMI or unpatched instance leaves the host exposed to the failure modes described in the advisory. Correlate yum history with system logs:

sudo yum history | head
sudo yum history list kernel-livepatch-5.10.186-179.751
sudo yum history info <id>
sudo ausearch -m AVC,USER_AVC -ts today | tail -100
cat /proc/sys/kernel/tainted   # non-zero = tainted kernel / out-of-tree modules

Quick Triage

Run these on Amazon Linux 2 to capture the current state of kernel-livepatch-5.10.186-179.751:

rpm -q kernel-livepatch-5.10.186-179.751                              # installed NVR
rpm -V kernel-livepatch-5.10.186-179.751                              # verify shipped files
sudo yum check-update --security
sudo yum updateinfo list cves
systemctl --failed --no-pager
sudo firewall-cmd --list-all 2>/dev/null || sudo iptables -L -n
getenforce && sestatus
# If kernel-livepatch-5.10.186-179.751 ships a systemd unit (unit name may differ from pkg name, e.g.
# bind→named, postgresql-server→postgresql, php-fpm→php-fpm):
systemctl list-unit-files | grep -i kernel | head

Step-by-Step Diagnosis

1. List failed systemd units.

   systemctl --failed --no-pager

2. Tail the journal for kernel-livepatch-5.10.186-179.751 and the system bus.

   sudo journalctl -u kernel-livepatch-5.10.186-179.751 -f --no-pager
   sudo journalctl -xe -f --no-pager

3. Inspect firewall / security-group posture from inside the instance.

   sudo firewall-cmd --list-all-zones --permanent 2>/dev/null || true
   sudo nft list ruleset 2>/dev/null | head -50
   ss -tulpen | head

4. Surface SELinux denials and author a local policy module if needed.

   sudo ausearch -m AVC,USER_AVC -ts today
   sudo ausearch -m AVC -ts today | audit2allow -a -M /tmp/local-fix
   sudo semodule -i /tmp/local-fix.pp

5. Verify kernel-livepatch-5.10.186-179.751 integrity and reinstall if anything is altered.

   sudo rpm -V kernel-livepatch-5.10.186-179.751
   sudo yum reinstall -y kernel-livepatch-5.10.186-179.751

6. Correlate findings with /var/log/yum.log, yum history, Amazon Inspector findings, and Amazon Linux advisory ALAS2LIVEPATCH-2023-155 to pin the change that introduced kernel-livepatch-5.10.186-179.751 — multiple vulnerabilities (9 CVEs) — patch and remediation guide.

Solution – Primary Fix

Apply the corrective yum transaction referenced by Amazon Linux advisory ALAS2LIVEPATCH-2023-155, then reload affected systemd units:

sudo yum -y makecache
sudo yum -y update --security              # apply ALL security errata (recommended)
# Or target a single package:
sudo yum -y update kernel-livepatch-5.10.186-179.751
sudo systemctl daemon-reload
# Unit name may differ from pkg name; check first:
systemctl list-unit-files | grep -i kernel | head
sudo systemctl restart kernel-livepatch-5.10.186-179.751
rpm -q kernel-livepatch-5.10.186-179.751                                # confirm new NVR
systemctl is-active kernel-livepatch-5.10.186-179.751 2>/dev/null       # confirm running (if a unit exists)

For kernel / glibc / systemd / openssl advisories a reboot is required (or Live Patching where available):

sudo needs-restarting -r                    # report whether reboot needed
sudo systemctl reboot                       # or: sudo shutdown -r now
# Amazon Linux Live Patching for the kernel (when enabled on the instance):
sudo yum install -y kernel-livepatch
sudo yum kernel-livepatch enable           # AL2 / AL2023
sudo yum kernel-livepatch status

Roll the same change across an Auto Scaling group / fleet with AWS Systems Manager Patch Manager:

aws ssm send-command 
  --document-name AWS-RunPatchBaseline 
  --targets Key=tag:Patch,Values=yes 
  --parameters 'Operation=Install,RebootOption=RebootIfNeeded' 
  --comment 'Apply Amazon Linux security errata'
aws ssm list-command-invocations --details --max-results 5
# Confirm the patch landed across the fleet:
aws ssm describe-instance-patch-states-for-patch-group 
  --patch-group <patch-group>

For immutable infrastructure, rebuild the golden AMI in EC2 Image Builder so newly launched instances start patched (do not rely on in-place patching alone):

aws imagebuilder start-image-pipeline-execution 
  --image-pipeline-arn arn:aws:imagebuilder:<region>:<acct>:image-pipeline/<name>
aws imagebuilder list-images --owner Self --max-results 5
# Then update the launch template / ASG to the new AMI:
aws ec2 create-launch-template-version --launch-template-id <lt-id> 
  --source-version '$Latest' --launch-template-data 'ImageId=<new-ami-id>'

Solution – Alternative Approaches

If the primary patch is not viable, choose from these:

• Roll back the offending yum transaction:

  sudo yum history list | head
  sudo yum history info <id>
  sudo yum history undo <id>

• Version-lock the package so yum cannot upgrade it:

  sudo yum install -y yum-plugin-versionlock
  sudo yum versionlock add kernel-livepatch-5.10.186-179.751
  sudo yum versionlock list
  sudo yum versionlock delete kernel-livepatch-5.10.186-179.751      # remove the lock

• Install an older NVR if a regression is suspected:

  yum --showduplicates list kernel-livepatch-5.10.186-179.751 | tac | head
  sudo yum install -y --allowerasing kernel-livepatch-5.10.186-179.751-<older-NVR>

• Switch SELinux to permissive briefly to confirm policy is the cause, then re-enforce:

  sudo setenforce 0
  # reproduce, capture denials, author a custom module:
  sudo ausearch -m AVC -ts recent | audit2allow -a -M mylocal
  sudo semodule -i mylocal.pp
  sudo setenforce 1

• Take an EBS snapshot of the root volume before kernel / glibc upgrades for fast rollback:

  INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
  VOL_ID=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID 
    --query 'Reservations[].Instances[].BlockDeviceMappings[?DeviceName==`/dev/xvda`].Ebs.VolumeId' 
    --output text)
  aws ec2 create-snapshot --volume-id $VOL_ID 
    --description 'pre-patch-snapshot' --tag-specifications 
    'ResourceType=snapshot,Tags=[{Key=Name,Value=pre-patch}]'

• For immutable workloads, swap the ASG to a previous AMI version instead of patching in place:

  aws ec2 describe-launch-template-versions --launch-template-id <lt-id> --max-results 5
  aws autoscaling update-auto-scaling-group --auto-scaling-group-name <asg> 
    --launch-template LaunchTemplateId=<lt-id>,Version='<prev-version>'
  aws autoscaling start-instance-refresh --auto-scaling-group-name <asg>

• Where Kernel Live Patching is enabled, apply kernel fixes without reboot:

  sudo yum kernel-livepatch status
  sudo yum kernel-livepatch enable
  sudo yum update -y --advisory=ALAS2LIVEPATCH-2023-155 kernel

Verification & Acceptance Criteria

All of these should pass after the fix:

rpm -q kernel-livepatch-5.10.186-179.751                                            # expected fixed NVR
sudo yum updateinfo list cves --installed              # CVEs above no longer listed
systemctl is-active kernel-livepatch-5.10.186-179.751 2>/dev/null
sudo journalctl -u kernel-livepatch-5.10.186-179.751 --since "5 minutes ago" --no-pager | grep -iE "error|fail" || echo OK
sudo firewall-cmd --list-services 2>/dev/null || sudo iptables -L -n
getenforce
sudo needs-restarting -r
# Inspector should drop the finding within ~24h:
aws inspector2 list-findings 
  --filter-criteria 'vulnerabilityId={comparison=EQUALS,value=CVE-2023-3777}'

The original reproduction for kernel-livepatch-5.10.186-179.751 — multiple vulnerabilities (9 CVEs) — patch and remediation guide must not trigger across two consecutive runs.

Rollback Plan

Capture state before any change:

rpm -qa > /root/rpm-pre.txt
sudo yum history list > /root/yum-history-pre.txt
# Optional pre-patch EBS snapshot of the root volume (run from inside the instance):
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
aws ec2 create-snapshot --volume-id <vol-id> --description 'pre-patch'

To revert if the patch is bad:

sudo yum history undo <id>
# Or downgrade just the package:
sudo yum install -y --allowerasing kernel-livepatch-5.10.186-179.751-<older-NVR>
sudo systemctl daemon-reload
sudo systemctl restart kernel-livepatch-5.10.186-179.751
# Or replace the instance from a snapshot/older AMI via ASG instance refresh:
aws autoscaling start-instance-refresh --auto-scaling-group-name <asg>
# Custom SELinux policy cleanup:
sudo semodule -r mylocal

Prevention & Hardening

Reduce the chance of this recurring on Amazon Linux 2:

• Enable automatic security patching on each instance:

  sudo yum install -y yum-cron
  sudo sed -i 's/^update_cmd.*/update_cmd = security/' /etc/yum-cron/yum-cron.conf 2>/dev/null || true
  sudo sed -i 's/^upgrade_type.*/upgrade_type = security/' /etc/dnf/automatic.conf 2>/dev/null || true
  sudo sed -i 's/^apply_updates.*/apply_updates = yes/' /etc/dnf/automatic.conf 2>/dev/null || true
  sudo systemctl enable --now yum-cron.service

• Drive fleet-wide patching through AWS Systems Manager Patch Manager (preferred for any fleet bigger than a handful of instances):

  aws ssm send-command --document-name AWS-RunPatchBaseline 
    --targets Key=tag:Patch,Values=yes 
    --parameters 'Operation=Install,RebootOption=RebootIfNeeded'
  aws ssm describe-patch-baselines --filters Key=OWNER,Values=AWS
  aws ssm get-default-patch-baseline --operating-system AMAZON_LINUX_2

• Enable Amazon Inspector for continuous CVE / package vulnerability scanning:

  aws inspector2 enable --resource-types EC2 ECR LAMBDA
  aws inspector2 list-findings 
    --filter-criteria 'severity={comparison=EQUALS,value=HIGH}' --max-results 20
  aws inspector2 batch-get-account-status

• Bake patched golden AMIs with EC2 Image Builder and roll them via ASG instance refresh instead of in-place patching for immutable infrastructure:

  aws imagebuilder list-image-pipelines
  aws imagebuilder start-image-pipeline-execution 
    --image-pipeline-arn arn:aws:imagebuilder:<region>:<acct>:image-pipeline/<name>
  aws autoscaling start-instance-refresh --auto-scaling-group-name <asg>

• Subscribe to alas.aws.amazon.com and watch AWS security bulletins for upstream changes.

• Version-lock sensitive packages so they cannot be auto-upgraded:

  sudo yum install -y yum-plugin-versionlock
  sudo yum versionlock add kernel-livepatch-5.10.186-179.751

• Monitor file integrity with AIDE:

  sudo yum install -y aide
  sudo aide --init && sudo mv /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz
  sudo aide --check

• Enable Kernel Live Patching so kernel CVEs can be remediated without reboot:

  sudo yum install -y kernel-livepatch
  sudo yum kernel-livepatch enable
  sudo yum kernel-livepatch status

• Keep SELinux in enforcing mode and review custom modules in /etc/selinux/targeted/ after every package upgrade.

• Apply CIS Amazon Linux 2 Benchmark hardening and remove unused packages.

Related Errors & Cross-Refs

Issues that commonly surface alongside kernel-livepatch-5.10.186-179.751 — multiple vulnerabilities (9 CVEs) — patch and remediation guide: yum lock contention, systemd unit ordering cycles, SELinux AVC bursts, security-group / NACL drift, and kernel taint flags after out-of-tree modules. Useful triage:

sudo yum check
systemd-analyze critical-chain
sudo ausearch -m AVC -ts today | tail
sudo firewall-cmd --get-active-zones 2>/dev/null || sudo iptables -L -n
cat /proc/sys/kernel/tainted
sudo needs-restarting -r
aws ssm describe-instance-patches --instance-id <i-xxxx> | tail -40

View all amazon-linux-2 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

Primary reference: Amazon Linux advisory ALAS2LIVEPATCH-2023-155. Manual pages useful on Amazon Linux 2:

man yum
man yum.conf
man systemctl
man journalctl
man firewall-cmd
man semanage
man audit2allow
man sosreport

Other resources: alas.aws.amazon.com, SSM Patch Manager docs, Amazon Inspector docs, EC2 Image Builder docs, and per-package notes in /usr/share/doc/kernel-livepatch-5.10.186-179.751/ for components implicated in kernel-livepatch-5.10.186-179.751 — multiple vulnerabilities (9 CVEs) — patch and remediation guide.

View all Amazon Linux 2 tutorials on the Tutorials Hub →