I have a similar setup. I use d.rymcg.tech (a configuration manager for Docker, as well as a collection of open source web services and config templates) and have Traefik (reverse proxy) on a Digital Ocean dropet connected to a VM in my home lab through wireguard. This framework allows me to put authentication and authoriation in front of any apps/services I'm hosting (HTTP basic auth, oauth2, mTLS). This setup allows me to control what is allowed access from outside of my home, without opening any ports.

How will running a CA limit access? eg. Do you want to do client side cert validation? That sounds like an overcomplication. Also not ideal to run a CA (have signing keys) on the proxy server.

Have you considered other approaches, such as Tailscale or Cloudflare Tunnels? I think you’re complicating things.

You can deal with the non-static IP by using duckdns.org

That sounds like a great plan, and a great way to learn how this sort of thing works.

it might be better to skip the cloud server and use cloudflare for dynamic dns. The standardized way to restrict access to websites is with client certificates or a basic authentication (user/pass) proxy. That would help avoid issues with internet traffic passing through the VPN accidentally.

If you use tailscale you can omit the raspberry pi and tunnel directly from the reverse proxy to each server (could do this with wireguard but requires a little more setup). Also you can configure your cloud server as an exit node so that all traffic from your device go through it, sort of as a vpn service. It's not as anonymous as a paid VPN service but at least you bypass the ISP or local wifi provider if you're out and about.

Well if you don't have a static IP, then your reverse proxy is going to break when your lease changes anyway. Not sure what your intended goal is for access and to what, but this is certainly a more costly and complex setup than needed for whatever it is.