Anyone got this to work between a linux host and a steamdeck?

I am using the arch package of steam on the host, I opened the necessary ports, but I can't get the two to download from each other (just to be safe I completely disabled the firewall on both devices and tested, doesn't work).

Steam's remote connection logs mention receiveing broadcast from the other device and when I start the download, the server briefly starts saying "hosting local file transfer" in its downloads page but stays in 0% a little while and disappears.