Affected versions: FreeBSD 12

📖 ~4 min read  •  Source: FreeBSD VuXML

VuXML topic: go — invalid headers are normalized, allowing request smuggling

Related CVEs: CVE-2019-16276

Upstream summary: The Go project reports: net/http (through net/textproto) used to accept and normalize invalid HTTP/1.1 headers with a space before the colon, in violation of RFC 7230. If a Go server is used behind a reverse proxy that accepts and forwards but doesn't normalize such invalid headers, the reverse proxy and the server can interpret the headers differently. This can lead to filter bypasses or request smuggling, the latter if requests from separate clients are multiplexed onto the

Table of contents
  1. Symptom & Impact
  2. Environment & Reproduction
  3. Root Cause Analysis
  4. Quick Triage
  5. Step-by-Step Diagnosis
  6. Solution – Primary Fix
  7. Solution – Alternative Approaches
  8. Verification & Acceptance Criteria
  9. Rollback Plan
  10. Prevention & Hardening
  11. Related Errors & Cross-Refs
  12. References & Further Reading

Symptom & Impact

go-devel on FreeBSD 12 runs only when something invokes it, so there is no crash loop and no failed unit to notice — the exposure is input-driven. A developer build, a CI job, a package rebuild or triage tooling that runs objdump, readelf or gdb across an untrusted binary can hit a malformed section header and corrupt memory inside the toolchain process, running code as the build user. On a shared builder or CI runner that means tampered build output rather than downtime. go-devel is a build-time developer tool that exits when the command finishes, so there is no service to restart — only in-flight builds and long-lived helpers such as distcc or clangd still hold the old binaries.

Environment & Reproduction

Reproduction targets FreeBSD 12. Confirm release, installed package, and capture baseline state:

freebsd-version -kru
uname -a
pkg info go-devel
pkg query "%n-%v" go-devel
pkg audit -F
service -e

Exercise the workload that uses go-devel while collecting:

tail -200 /var/log/messages
dmesg -a | tail -200
tail -200 /var/log/pkg.log

Root Cause Analysis

Root cause is tracked at FreeBSD VuXML. The FreeBSD ports security team shipped a corrective go-devel port revision; hosts on an outdated build remain exposed. Correlate package logs with system logs and kernel state to isolate the change that triggered the failure mode:

tail -500 /var/log/pkg.log
tail -500 /var/log/messages
sysctl kern.lastpid
sysctl kern.osreldate     # numeric __FreeBSD_version, e.g. 1400097

Quick Triage

Run these checks on FreeBSD 12 to confirm the failure mode and current state of go-devel:

pkg version -v go-devel                # installed vs available version
pkg audit go-devel                     # advisory match for this package
tail -100 /var/log/messages
dmesg -a | tail -100
kldstat                              # kernel module state (for kernel/driver pkgs)
pfctl -sr 2>/dev/null || ipfw list   # only if pf/ipfw is enabled
# e.g. bind918→named, php83→php-fpm), check it:
pkg query '%Fp' go-devel | sed -n 's|.*/etc/rc\.d/||p'  # FreeBSD: the REAL service name(s) = basename of the rc.d script (bind918 -> named, postgresql16-server -> postgresql, apache24 -> apache24)
pkg query '%b' go-devel  # FreeBSD: shared libraries PROVIDED by go-devel. Non-empty here + no rc.d script = it is a library: patch it, then restart its consumers

Step-by-Step Diagnosis

  1. List enabled services (only relevant if the package provides one).

    service -e
  2. Follow live logs.

    tail -F /var/log/messages
    dmesg
  3. Validate firewall rules (skip if neither pf nor ipfw is enabled).

    pfctl -sr -v 2>/dev/null || ipfw show
  4. Check package integrity for go-devel.

    pkg check -B go-devel
    pkg check -d go-devel    # verify shared-library deps
  5. Reinstall go-devel if integrity check fails.

    pkg install -fy go-devel
  6. Correlate findings with /var/log/pkg.log and FreeBSD VuXML to pin the commit that introduced the regression.

Solution – Primary Fix

Install the corrective go-devel port revision referenced by FreeBSD VuXML:

sudo pkg update
sudo pkg upgrade go-devel              # or: sudo pkg upgrade -y for the whole system
pkg audit go-devel                    # confirm no remaining advisory for this package

For ports-tree builders (FreeBSD 13.x and earlier used portsnap; on FreeBSD 14+ the ports tree is fetched with Git):

# FreeBSD 14+ (portsnap was removed):
sudo pkg install -y git-lite
sudo git clone --depth 1 https://git.FreeBSD.org/ports.git /usr/ports
# FreeBSD 13.x and earlier:
# sudo portsnap fetch update
cd /usr/ports/<category>/go-devel
sudo make deinstall reinstall clean

Reboot only if the package ships a kernel module or replaces a shared library used by long-running daemons.

Need help rolling this patch across a FreeBSD fleet? Our IT Solutions & Services team manages FreeBSD jail/bhyve patch windows. Get in touch for a free consultation.

Solution – Alternative Approaches

If the primary fix is not viable, choose from these alternatives:

  • Lock the package until the fix is vetted:

    sudo pkg lock go-devel
  • Downgrade to a known-good revision. pkg install pkgname-VERSION is not a real downgrade syntax — fetch a specific build instead:

    # 1. Discover available versions across configured repos:
    pkg search -e go-devel
    pkg rquery -r FreeBSD-quarterly '%n-%v' go-devel
    # 2. Install from a specific saved .pkg file:
    sudo pkg add -f /path/to/go-devel-<older-version>.pkg
    # 3. Or switch the host repo to the quarterly branch (see snippet below) and:
    sudo pkg upgrade -fr FreeBSD-quarterly go-devel
  • Switch the pkg repository between quarterly and latest by editing /usr/local/etc/pkg/repos/FreeBSD.conf:

    FreeBSD: {
      url: "pkg+https://pkg.FreeBSD.org/${ABI}/quarterly",
      mirror_type: "srv",
      signature_type: "fingerprints",
      fingerprints: "/usr/share/keys/pkg",
      enabled: yes
    }
  • Isolate the affected service in a jail with stricter firewall rules:

    iocage create -n go-devel-jail -r 12.4-RELEASE
    iocage set allow_raw_sockets=0 go-devel-jail
    # or with Bastille:
    bastille create go-devel-jail 12.4-RELEASE 10.0.0.10
  • Replace the service with a vendored static build for the period between exposure detection and full rollout.

Verification & Acceptance Criteria

All of these should pass after the fix:

pkg info go-devel                # shows the expected fixed version
pkg audit go-devel               # no advisory for this package (exit code 0)
tail -50 /var/log/messages   # no new errors after upgrade
# If go-devel ships a service, confirm it is running under its rc.d name:

The conditions described in the advisory must no longer be reported for go-devel across two consecutive runs.

Rollback Plan

Capture state before any change (only ZFS root has boot environments — UFS hosts skip bectl):

pkg query "%n-%v" > /root/pkg-pre.txt
# ZFS-on-root only:
sudo bectl create pre-go-devel-patch

To revert if the upgrade is bad, reinstall the previously saved .pkg file:

sudo pkg add -f /var/cache/pkg/go-devel-<previous-version>.pkg
# Or activate the pre-patch boot environment and reboot (ZFS-on-root only):
sudo bectl activate pre-go-devel-patch
sudo shutdown -r now

For kernel/loader changes on a UFS host, boot the previous kernel from the loader prompt (press 3 at the menu, then boot kernel.old).

Prevention & Hardening

Prevent recurrence on FreeBSD 12 hosts running go-devel:

  • Enable the daily security pkg audit in /etc/periodic.conf:

    daily_status_security_pkgaudit_enable="YES"
  • Subscribe to freebsd-security-notifications at lists.freebsd.org.

  • Mirror through a local pkg repository managed by poudriere:

    poudriere jail -c -j 12amd64 -v 12.4-RELEASE
    poudriere ports -c -p default
    poudriere bulk -j 12amd64 -p default <category>/go-devel
  • Version-pin sensitive packages:

    sudo pkg lock go-devel
  • Take an automatic ZFS boot-environment snapshot before every upgrade (ZFS root only):

    sudo bectl create pre-upgrade-$(date +%Y%m%d)
  • Monitor file integrity (create a baseline, verify against it later):

    # Create a baseline (use -c; target /usr/local/etc, /etc, /boot — NOT /):
    sudo mtree -c -K sha256digest -p /usr/local/etc > /var/db/usr-local-etc.mtree
    sudo mtree -c -K sha256digest -p /etc          > /var/db/etc.mtree
    # Verify later:
    sudo mtree -p /usr/local/etc < /var/db/usr-local-etc.mtree
    # Or with AIDE for a richer ruleset:
    sudo pkg install -y aide && sudo aide --init && sudo aide --check
  • Harden jails with allow.* tunables in /etc/jail.conf:

    go-devel_jail {
      allow.raw_sockets = 0;
      allow.sysvipc    = 0;
      allow.mount      = 0;
      allow.chflags    = 0;
    }

Issues that commonly surface alongside a go-devel update: pkg lock contention, mismatched ABI after kernel/userland skew, pf rule drift, and stale shared-library references after upgrade. Triage with:

freebsd-version -kru
uname -K
pkg check -d
pfctl -sr

View all freebsd-12 tutorials on the Tutorials Hub →

Browse all common problems & solutions on the Tutorials Hub.

References & Further Reading

Primary reference: FreeBSD VuXML. Useful manual pages on FreeBSD 12:

man pkg
man freebsd-update
man pfctl
man ipfw
man bectl
man periodic.conf

Other resources: the FreeBSD Handbook, the FreeBSD Security Advisories at security.freebsd.org, and the /usr/ports/UPDATING file for port-specific notes implicated in this advisory.