Part of the SoC makes a lot of sense but I'd still like to have an expansion option. Or, well, actually, maybe connecting it up via PCIe might be sufficient, latency is going to take a serious hit but if there's gigabytes of HBM on the chip acting essentially as cache it's probably fine for pretty much all practical workloads. Gigantic memory requirements don't tend to come with purely random access patterns.

OTOH that definitely puts the "N" in "NUMA". I doubt any OS but Linux could deal with the thing sanely.