https://stackoverflow.com/questions/3287038/cron-and-virtualenv

I haven't looked at the particulars of this applications, but if you path the python binary you use to run the application, it should use the environment that's with it, without being activated. Activate just prepends the path for that venv to every command you give from then on when you're working in a shell. And as noted in there, make sure you specify /bin/bash as your shell in cronjobs since it uses sh by default so you might run into issues in that context.

A virtual environment is just a copy of the python and pip binaries. When you activate the venv, the venv dirs temporarily get added to your path, so your regular python alias points to the binary in the venv (run which python with venv active to verify). Pip will install modules to a subdir of your venv. It basically works like npm and the node_modules dir.

Docker doesnt replace your current system. It just runs containers (which act like a separate system)

You can also try podman which wont silently rewrite your firewall rules without telling you... I'll never forgive docker for doing that

I think your failings with docker stem from a complete lack of understanding. It takes little to no effort and will replace nothing. You're causing yourself a lot more work, not just now but also in the future, trying to do things the way you are. It shouldn't take more than 10 minutes to install docker, docker compose, and research the setting you need to add to a compose file before running it. At that point you're done, no dependencies, no maintenance. Need to update? Pull the new image and relaunch the container. Takes seconds.

I can't imagine trying to wrap my head around python dependencies and venv when docker seems a bridge too far.

Okay, so I think I can help with this a little.

The "secret sauce" of Docker / containers is that they're very good at essentially lying to the contents of the container and making it think it has a whole machine to itself. By that I mean the processes running in a container will write to say /config and be quite content to write to that directory but docker is secretly redirecting that write to somewhere else. Where that "somewhere else" is, is known as a "volume" in docker terminology and you can tell it exactly where you want that volume to be. When you see a command with -vin it, that's a volume map - so if you see something like -v /mnt/some/directory:/config in there - that's telling docker "when this container tries to write to /config, redirect it to /mnt/some/directory instead.

That way you can have 10 containers all thinking they're each writing to their own special /config folder but actually they can all be writing to somewhere unique that you specify. That's how you get the container to read and write to files in specific locations you care about, that you can backup and access. That's how you get persistence.

There's other ways of specifying "volumes", like named volumes and such but don't worry too much about those, the good ol' host path mapping is all you need in 99% of cases.

If you don't specify a volume, docker will create one for you so the data can be written somewhere but do not rely on this - that's how you lose data, because you'll invariably run some docker clean command to recover space and delete an unused unnamed volume that had some important data in it.

It's exactly the same way docker does networking, around port mapping - you can map any port on your host to the port the container cares about. So a container can be listening on port 80 but actually it's being silently redirected by the docker engine to port 8123 on your host using the -p 8123:80 argument.

Now, as for updates - once you've got your volumes mapped (and the number and location of them will depend on the container itself - but they're usually very well documented), the application running in the container will be writing whatever persistence data it needs to those folders. To update the application, you just need to pull a newer version of the docker container, then stop the old one and start it again - it'll start up using the "new" container. How well updates work really depends on the application itself at this point, it's not really something docker has any control over but the same would be if you were running via LXC or apt-get or whatever - the application will start up, read the files and hopefully handle whatever migrations and updates it needs to do.

It's worth knowing that with docker containers, they usually have labels and tags that let you specify a specific version if you don't want it updating. The default is an implied :latest tag but for something like postgress which has a slightly more involved update process you will want to use a specific tag like postgres:14.3 or whatever.

Hope that helps!