If you want embedded boards Rockchip and Sunxi/AllWinner are pretty well supported by the Linux kernel. Go have a look at boards with full Armbian support, that's usually a good shortcut to finding one.

My preference runs to the Nanopi boards, they're better built than Orange Pi hardware. You're going to see a lot of Orange Pi recommendations based on cost but be aware that they're not all that well made and occasionally have reliability problems. I was pretty chuffed for my $20 Orange Pi zero until I realized that the WiFi basically had zero chance of working reliably. Pick models carefully after reading about people's experiences with them on the Armbian forums so you can avoid duds.

If you don't need embedded arm check out the thin client selection on eBay. You can buy a J5005 Dell/Wyse thin client for like $100, some models even have a low profile pcie slot (these cost a bit more because they're desirable as pf/OPNsense platforms.) These make pretty solid Proxmox or container host platforms, or you can use them for their intended purpose and jam in a low profile graphics card.

My personal "I don't feel like spending $150 on a 4gb pi" favorite is the HP T630 thin client. On a good day you'll find an 8gb RAM model with the power brick for <$60 shipped. Do the eBay thing with any of these and try to best offer the price down a bit if it's an option.

If you want to step up a notch check out the HP T730, this one comes with a pcie slot and makes a fairly decent Proxmox virtualized router host. They're usually available for <$130/shipped or less. The HP T740 is the same thing with a Zen1 embedded SoC, those run ~$220 or so and support NVMe. The Wyse 5070 offers Celeron or Pentium options and is a <10W machine, the J5005 version actually works pretty well as a hardware transcoding PLEX host (provided you're not transcoding 4k.)

The T730 and T630 use SATA m.2 storage, the 5070 and T640 support NVMe. All of these have an m.2 A+E key slot for WiFi or an extra 2230 NIC.