I fully understand. :)

For hosting providers, you can always scout out deals on lowendbox.com or webhostingtalk.com, especially this time of year where black Friday deals are coming. Then, install something like HestiaCP (https://hesticp.com) and it will take care of the rest of the stuff like email and site hosting. Plus - with a VPS, you get a shiny new static IP you can use. :)

If you don't want a site, that's easy enough, just make an empty index.html page so if a curious email recipient wants to visit your site based on your email's domain, they just get a blank page and move on.

Either that - go with a well known hosting provider which does basic cPanel hosting with low disk space.

I've never dared going the route of setting up an e-mail server at home mostly because of having a dynamic IP which rotates often enough to cause problems.

I have had pretty good experience with hosting an email server on AlphaVPS, InceptionHosting and just now GreenCloudVPS.

GreenCloudVPS currently have a promotion until Sunday, and there are usually promotions around Black Friday on LowEndSpirit and LowEndTalk