it just seems to redirect to an otherwise Internet accessible page.

I'm using authelia with caddy but I'm guessing it could be similar, you need to configure the reverse proxy to expect the token the authentication service adds to each request and redirect to sign in if not. This way all requests to the site are protected (of course you'll need to be aware of APIs or similar non-ui requests)

I have to make an Internet accessible subdomain.

That's true, but you don't have to expose the actual services you're running. An easy solution would be to name it other thing, specially if the people using it trust you.
Another would be to create a wildcard certificate, this way only you and those you share your site with will know the actual sub domain being used.

My advice is from my personal setup, but still all internal being able to remotely access it via tailscale, so do you really need to make your site public to the internet?
Only if you need to share it with multiple people is worth having it public, for just you or a few people is not worth the hassle.