The way I see it, the software can be open source, but you'd have to train it yourself.

Kind of like how you're free to reverse engineer a console, and write an open source emulator, but you can't supply the firmware itself (ex scph1000.bin for ps1) or roms of commercial games.

The pretrained part is just someone running the software on their dataset for you. You are free to do the same yourself, and getting the data for the training set legally is an exercise for you. Is it affordable for most people? Not really, because you need gargantuan amounts of data and compute power. But the software itself is yours to modify and run. I see that as an indication of the technology being a dead end, in the long run. As in, they are not getting much better, but they are becoming much larger and much less feasible to train.