No worries! Writing that down actually helped clarify some of my thoughts.
Something extra: distributed computing.
Let's say you have 3 processes that need to communicate with one another. There's heaps of tooling available in OSs to manage those processes. Logging, networking, filesystem access, privilege separation, resource allocation... all provided by the host OS without installing anything. But what if those 3 processes can't run on one "machine"? Which process should go where? What if it needs 8GB memory but there's only 6GB available on some of the machines? Who controls that?
Systems like Kubernetes, Nomad, Docker Swarm etc. offer a way to manage this. They let us say something like:
- run this process (by specifying a container image),
- give it at least these resources (xGB memory, xvCPUs)
- let it communicate with these other processes (e.g. pods, overlay networks...)
These systems manage containers. If you want to do distributed computing and want to take advantage of those systems to manage it, stuff needs to be run in containers.
Containers are not the only way to do distributed computing - far from it! But over the past few years this particular approach has become popular in the, umm... "commercial software development industry".
Opinion. Are Linux containers something to look into as someone who doesn't work in the industry? Unless you're interested in how containers themselves work and/or distributed computing; frankly - no. Computers are still getting faster and cheaper. So why is all this stuff so popular in the commercial world? I'll end with some tongue-in-cheek.
Partly it's because the software development industry is made up of actual human beings who have their own emotions and desires. Distributed computing is a fun idea because tech people are faced with challenges tech people are interested in.
Boring: can we increase our real estate agency brand recognition by 200%? We could provide property listings as both a CSV and PDF to our partners! Our logo could go on the PDF! Wow! Who knows how popular our brand could be?
Fun: can we increase throughput in this part of the system by 200%? We might need to break that component out to run on a separate machine! Wow! Who knows who fast it could go?
There's some new tech called "Large Language Models". Apparently this lets people, including programmers, work way faster. These so-called LLMs can ingest our own natural instructions, like "make me a UI which is not totally fucked", or "refactor this dogshit code which we only keep around because it makes us more money than we know what to do with"! Not only that, the LLM will actually respond with code!
And more code is exactly what the software industry - nay, the whole entire world - needs. Astonishing.
I'm 100000% confident that Microsoft has not heard of this amazing tech, otherwise we would not see such a total shitshow.