Maybe try out FreedomBox? freedombox is a Debian package which automatically sets up apache2, firewalld, fail2ban and Letʼs Encrypt. It also automatically adds pre-canned configuration files for applications you install with it (e.g. Mediawiki, WordPress, Matrix, Postfix/Dovecot). The theoretical goal of FreedomBox is to allow anyone to set up a webserver and administer it via a webUI. So, although I would say itʼs not quite there yet for command-line-illiterate users, I have found the software useful as a turnkey server to see what makes certain web applications tick, albeït in mostly vanilla form.

For example, after installing a new app like WordPress, you could examine what exactly the FreedomBox scripts changed in the /etc/apache2/ or /etc/fail2ban/ configuration files.

Changing the port seems like a pointless step, just disallow access from everywhere and allow only from select IPs. Port scanners will scan all open ports and will detect that it's ssh, regardless of port number.

Your plan is solid. The important thing is that you configure those things correctly, but you’re following guides so that should be ok. It’s on a VPS so there’s no threat to your home network, and none of those services pose a significant risk to you even if they were compromised so there’s no reason to go overboard.

If I had any further advice to give it’d be:

• Change any default usernames and passwords that any of your apps/databases use.

• Use randomly generated passwords for all service accounts. So that if you do find yourself compromised, they don’t then know a password that you’ve reused somewhere else (like your email account).

• Run those services using something like Docker with no access to each other.

• Utilize your VPS provider’s cloud firewall if they have one. If you’re paying for a cheap VM, it shouldn’t need to deal with all the general firewalling from the internet. VPS providers often have free cloud firewalls you can offload that work to.