The reason you can't is "because Intel deliberately designed it that way". Back when USB was just a notion, PDAs were a really cool thing. There was apparently concern at Intel that someday these little things might be all that someone might own. You might connect your PDA directly to the printer, rather than syncing it to your Intel Desktop and printing from there. You might connect your PDA to the modem and collect electronic mailographs directly, instead of syncing with a PC. If you could do enough without the PC middleman, you might even skip on buying an Intel computer altogether.

So, Intel baked into the protocol anything they could think of to make peer-to-peer communications impossible in USB, make life easy for the singular PC communications master, and put a timing onus on devices that forced them to be dumbed-down state machines instead of computers in their own right.