You'd do most of that stuff inside a container (Distrobox probably). You'd basically have a "clean OS" to start with (doesn't have to be the same OS as the host even) and install your libraries like normal. Distrobox does a good job of integrating with the host so you mostly won't know you're in a container. It's not perfect though, and if you have little experience with containers you'll definitely have a hard time doing what you need to.