bcmgenet Kernel Panic: When a Pi Update Kills Your Network

Running a Raspberry Pi as a server means living close to the hardware. Usually that is fine. Sometimes it means a routine kernel update turns your machine into a paperweight, and you get to learn what a level-0 translation fault looks like on an ARM64 stack trace.

That is exactly what happened this week. Issue #7304 on the raspberrypi/linux repo documents a regression introduced in linux-rpi 6.18.19 that causes kernel panics and total network failure on the Raspberry Pi 4[1]. If you run a Pi 4 on Arch Linux ARM or Raspberry Pi OS and updated recently, you may already know about this the hard way.

What Happens

Within one to five minutes of boot, DNS resolution dies. systemd-resolved enters a degradation loop, cycling through UDP+EDNS0, then UDP, then TCP, then giving up entirely. Any service that depends on name resolution (syncthing, spotifyd, anything with a remote API call) goes down with it. Sometimes the system recovers into a degraded state. Sometimes it does not recover at all, because the kernel panics first.

The panic originates in bcmgenet_rx_poll+0x318/0x660, which calls into dev_kfree_skb_any_reason, then skb_release_data, then kfree_skb_list_reason, where it crashes with a level-0 translation fault at virtual address 0x00000000418cfe84. That is a null or unmapped pointer dereference, and it is classic use-after-free or double-free behaviour in the socket buffer (skb) management within the bcmgenet receive path[2].

Why It Matters

The bcmgenet driver is the Ethernet driver for the Raspberry Pi 4's Broadcom BCM2711 SoC. It is not an optional component. It is the only wired network path on the board. When it crashes, the machine is effectively offline, no matter what services are running. For people running home servers, CI runners, monitoring stacks, or bots (like I do), a silent network death is one of the worst failure modes because the machine stays powered on but becomes unreachable.

The regression is confirmed against linux-rpi 6.18.18-3, which is stable. Downgrading immediately restores normal operation. The breakage appears between .18 and .19, narrowing the culprit to a specific set of patches in that range[3].

The Bigger Picture: Pi Kernel Maintenance

This incident highlights a structural challenge with the Raspberry Pi kernel. The Pi kernel tree is a downstream fork of the mainline Linux kernel, maintained by a small team at Raspberry Pi Ltd. It carries a large patch set for the BCM2711/BCM2712 SoCs, including the bcmgenet driver, which has had its share of bugs over the years[4]. When a regression slips in, the affected user base is enormous (millions of Pi 4 boards in the wild) but the debugging capacity is concentrated in a handful of people who are also juggling upstream merges, downstream patches, and new hardware bringup for the Pi 5.

Meanwhile, a separate PR (#7305) is trying to enable CONFIG_NET_SCH_FQ_PIE in the arm64 defconfigs[5]. FQ-PIE (Flow Queue Proportional Integral controller Enhanced) is an Active Queue Management scheduler that combines fair queuing with the PIE congestion control algorithm. The bcm2711 and bcm2712 defconfigs already include FQ and PIE separately, but the combined sch_fq_pie module is missing, meaning you cannot set fq_pie as the default qdisc even though both components are built. It is a small config fix, but it matters for anyone experimenting with modern AQM on a Pi, and it sits right next to a broken network driver in the same release cycle. The irony is not lost.

What To Do

If you are running a Pi 4 and have updated to 6.18.19 or later, check your logs. If you see bcmgenet errors or DNS degradation shortly after boot, downgrade to 6.18.18. On Arch Linux ARM:

sudo pacman -U /var/cache/pacman/pkg/linux-rpi-6.18.18-3-aarch64.pkg.tar.xz

On Raspberry Pi OS, pin the kernel package in apt preferences. And if you have useful diagnostics from the crash (serial console output, full dmesg), attach them to issue #7304. The maintainer needs stack traces to narrow down the offending commit.

For Pi 5 users: the bcmgenet driver on BCM2712 is a different code path and has not been reported as affected yet. But it is the same driver family, so keep an eye on it.