Affected versions: Oracle Linux 8

📖 ~4 min read  •  Source: ELSA advisory ELSA-2023-12312

Table of contents
  1. Symptom & Impact
  2. Environment & Reproduction
  3. Root Cause Analysis
  4. Quick Triage
  5. Step-by-Step Diagnosis
  6. Solution – Primary Fix
  7. Solution – Alternative Approaches
  8. Verification & Acceptance Criteria
  9. Rollback Plan
  10. Prevention & Hardening
  11. Related Errors & Cross-Refs
  12. References & Further Reading

Symptom & Impact

GraalVM is executed by every process started from it, so a flaw here surfaces inside your applications rather than in GraalVM itself: header-parsing and request-smuggling bugs in the bundled HTTP stack, weaknesses in the runtime’s own TLS and crypto code, unsafe archive, XML or deserialisation handling in the standard library, or memory-safety bugs in the JIT. Long-lived workers such as application servers, JVM daemons and resident node processes keep executing the old build after Oracle Linux 8 replaces it on disk, so a patched machine stays exposed until each one is restarted. GraalVM is an interpreter or virtual machine plus its standard library, not a daemon: it runs only inside processes you start, so there is no GraalVM unit to restart — restart the applications that execute it instead.

Environment & Reproduction

Reproduction targets Oracle Linux 8 running either the Red Hat Compatible Kernel (RHCK) or the Unbreakable Enterprise Kernel (UEK). Confirm release, kernel, and installed package:

cat /etc/oracle-release
uname -r
rpm -q GraalVM
dnf list installed GraalVM
dnf history list --reverse | head -20

Exercise the workload that uses GraalVM while collecting:

sudo needs-restarting -u  # --useronly: restrict to the invoking user's processes (useful on shared/app hosts)
sudo lsof -n +L1  # link count < 1 = file deleted but still open/mmap'd; this is the ground truth after an RPM replaces a .so (package: lsof)
sudo lsof -n +L1 2>/dev/null | awk 'NR>1 {print $1, $2}' | sort -u  # condensed COMMAND + PID list of every process still running old code
journalctl -xe --no-pager | tail -200
tail -200 /var/log/dnf.log
tail -200 /var/log/audit/audit.log

Root Cause Analysis

Root cause is documented in ELSA advisory ELSA-2023-12312. Upstream Red Hat / Oracle Linux maintainers shipped a fix in the corresponding GraalVM errata; running an outdated build leaves the host exposed and may trigger the failure modes described in the advisory. Correlate transaction history with system logs and SELinux audit entries to isolate the originating change:

dnf history list --reverse | head -30
dnf history info $(dnf history list | awk '/GraalVM/ {print $1; exit}')
ausearch -m AVC,USER_AVC -ts today | tail -100
cat /proc/sys/kernel/tainted   # non-zero = kernel modules / out-of-tree drivers loaded

Quick Triage

Run these checks on Oracle Linux 8 to confirm the failure mode and current state of GraalVM:

rpm -q GraalVM                              # installed version
rpm -V GraalVM                              # verify file integrity
dnf updateinfo info --security GraalVM     # any security advisories outstanding
systemctl --failed --no-pager
firewall-cmd --list-all 2>/dev/null || echo 'firewalld not running'
getenforce                                 # SELinux mode
# e.g. httpd pkg/unit match, but bind→named, postgresql-server→postgresql):
dnf repoquery -l GraalVM 2>/dev/null | grep -E '/lib/systemd/system/.*\.(service|socket|timer)$'  # same check WITHOUT installing (EL7/Amazon Linux 2: yum install yum-utils, then: repoquery -l GraalVM)
systemctl show --no-pager -p Type,RemainAfterExit <UNIT>  # Type=oneshot (often paired with a .timer) means a one-shot job, NOT a daemon -- do not 'restart' it, let the timer fire

Step-by-Step Diagnosis

  1. List failed units.

    systemctl --failed --no-pager
  2. Follow the journal for GraalVM and the system bus.

    journalctl -xe -f --no-pager
  3. Check firewall posture (skip if firewalld is masked).

    firewall-cmd --list-all-zones --permanent
    nft list ruleset 2>/dev/null | head -50
  4. Surface SELinux denials and translate them to a policy module if needed.

    ausearch -m AVC,USER_AVC -ts today
    ausearch -m AVC -ts today | audit2allow -a -M /tmp/local-fix
    # Inspect /tmp/local-fix.te before applying:
    sudo semodule -i /tmp/local-fix.pp
  5. Verify GraalVM integrity and dependency closure.

    dnf check
    rpm -V GraalVM
    rpm -q --requires GraalVM | xargs -r rpm -q --whatprovides | head
  6. Correlate findings with /var/log/dnf.log, dnf history and ELSA advisory ELSA-2023-12312 to pin the change that introduced the regression.

Solution – Primary Fix

Apply the corrective dnf transaction referenced by ELSA advisory ELSA-2023-12312, reload affected systemd units, and reconcile firewalld / SELinux state:

sudo dnf clean expire-cache
sudo dnf -y update GraalVM
sudo systemctl daemon-reload
sudo needs-restarting -s | sort -u  # STEP 1: read the list before acting. dbus.service, systemd-logind.service and PID 1 must NOT be blind-restarted on a live host -- those mean 'reboot'
sudo systemctl daemon-reload  # STEP 3: pick up any unit files the update rewrote
sudo systemctl restart <UNIT> && sudo systemctl is-active <UNIT> && sudo systemctl status --no-pager --full <UNIT>  # single-unit path, with an immediate health check
systemctl --user list-units --type=service --no-pager  # desktop/session and per-user services are a separate manager; restart them with: systemctl --user restart <UNIT>
rpm -q GraalVM                       # confirm new NVR

If the advisory says a reboot is required (kernel, glibc, systemd, openssl):

sudo needs-restarting -r           # reports kernel/init/glibc need
sudo systemctl reboot              # or: sudo shutdown -r now

Need help applying this fix at scale? Our IT Solutions & Services team rolls Oracle Linux patches across estates with zero-downtime change windows and Ksplice live-patching. Get in touch for a free consultation.

Solution – Alternative Approaches

If the primary fix is not viable, choose from these alternatives:

  • Roll back the offending dnf transaction:

    sudo dnf history list --reverse
    sudo dnf history undo <id>
  • Pin GraalVM with the versionlock plugin:

    sudo dnf install -y python3-dnf-plugin-versionlock
    sudo dnf versionlock add GraalVM
    sudo dnf versionlock list | grep GraalVM
  • Downgrade to a known-good NVR from the repo cache or vault:

    sudo dnf --showduplicates list GraalVM
    sudo dnf -y downgrade GraalVM-<older-NVR>
  • Switch firewalld backend (nftables ↔ iptables) for compatibility:

    sudo sed -i 's/^FirewallBackend=.*/FirewallBackend=iptables/' /etc/firewalld/firewalld.conf
    sudo systemctl restart firewalld
  • If SELinux is suspected, switch to permissive briefly, capture denials, and author a custom module before re-enforcing:

    sudo setenforce 0                                    # do NOT leave permissive
    # reproduce the failure
    sudo ausearch -m AVC -ts recent | audit2allow -a -M mylocal
    sudo semodule -i mylocal.pp
    sudo setenforce 1
  • Where the advisory has Ksplice coverage, live-patch without reboot (Oracle Linux Premier Support):

    sudo uptrack-show                # current live patches
    sudo uptrack-upgrade -y          # apply all available
    uptrack-uname -r                 # effective kernel after live patching

Verification & Acceptance Criteria

All of these should pass after the fix:

rpm -q GraalVM                                       # shows the expected fixed NVR
dnf updateinfo list --security installed | head    # no security advisories pending for us
firewall-cmd --list-services                        # required services present
getenforce                                          # intended mode (Enforcing/Permissive)

The conditions described in the advisory must no longer be reported for GraalVM across two consecutive runs.

Rollback Plan

Capture state before any change:

rpm -qa > /root/rpm-pre.txt
dnf history list --reverse > /root/dnf-history-pre.txt
# LVM snapshot of the root LV (size to ~10% of root):
sudo lvcreate -L 4G -s -n root_pre_patch /dev/mapper/$(lvs --noheadings -o lv_path | grep -m1 root | xargs basename)

To revert if the patch is bad:

sudo dnf history undo <id>
# Or downgrade just GraalVM to the previous NVR:
sudo dnf -y downgrade GraalVM
sudo systemctl daemon-reload
# For SELinux module additions:
sudo semodule -r mylocal
# Reboot only if kernel/initramfs/glibc were rolled back:
sudo systemctl reboot

For kernel rollbacks, select the previous entry from the GRUB menu or set it as default with grubby --set-default /boot/vmlinuz-<older>.

Prevention & Hardening

Prevent recurrence on Oracle Linux 8 hosts running GraalVM:

  • Enable scheduled security updates via dnf-automatic:

    sudo dnf install -y dnf-automatic
    # Edit /etc/dnf/automatic.conf:
    # upgrade_type = security
    # apply_updates = yes
    sudo systemctl enable --now dnf-automatic.timer
  • Subscribe to the Oracle Linux Errata RSS / mailing list at linux.oracle.com/security.

  • Mirror through a local yum/dnf repository:

    sudo dnf install -y dnf-utils createrepo_c
    sudo reposync --download-metadata --downloadcomps -p /srv/repos -m --repo=ol8_baseos_latest
    sudo createrepo_c /srv/repos/ol-baseos
  • Pin sensitive packages so they cannot be auto-upgraded:

    sudo dnf install -y python3-dnf-plugin-versionlock
    sudo dnf versionlock add GraalVM
  • Snapshot the root LV before every upgrade window:

    sudo lvcreate -L 4G -s -n root_pre_$(date +%Y%m%d) /dev/<vg>/<root-lv>
  • Monitor file integrity with AIDE:

    sudo dnf install -y aide
    sudo aide --init && sudo mv /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz
    sudo aide --check
  • Lock down with SELinux audit rules in /etc/audit/rules.d/:

    # /etc/audit/rules.d/90-cp.rules
    -w /etc/passwd  -p wa -k identity
    -w /etc/shadow  -p wa -k identity
    -w /etc/sudoers -p wa -k privilege
    -a always,exit -F arch=b64 -S execve -k exec
  • Where licensed, enable Oracle Ksplice for live kernel and userspace patching:

    sudo dnf install -y uptrack ksplice-tools
    sudo uptrack-upgrade -y
    sudo systemctl enable --now uptrack

Issues that commonly surface alongside a GraalVM update: dnf transaction lock contention, systemd unit ordering cycles, SELinux AVC bursts, firewalld zone drift, and kernel taint flags. Triage with:

cat /proc/sys/kernel/tainted
systemd-analyze critical-chain
ausearch -m AVC -ts today | tail
firewall-cmd --get-active-zones
dnf history list --reverse | head

View all oracle-linux-8 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

Primary reference: ELSA advisory ELSA-2023-12312. Useful manual pages on Oracle Linux 8:

man dnf
man dnf.conf
man systemctl
man journalctl
man firewall-cmd
man semanage
man audit2allow
man grubby

Other resources: Oracle Linux 8 Administrator’s Guide at docs.oracle.com, the upstream Red Hat CVE database at access.redhat.com/security/cve, the Oracle Ksplice known-fixes feed, and /usr/share/doc/GraalVM/ for component-specific notes implicated in GraalVM.