No PGP support

Why would the mail provider need to support it? I mean, if they provide some sort of webmail client, maybe it doesn't do PGP, but I sure wouldn't be giving them my PGP keys anyway.

I haven't used any of them, but I don't think that you can go too far wrong here, since you have your own domain. Pick one, try it for non-critical stuff for a month or two, and if you don't like it, switch. As long as you own the domain, you're not locked in. If you do like it, then just start migrating.

The main differentiating factors I can think of are (a) service reliability, (b) risk that someone breaks in and dumps client mail, but it's hard for me to evaluate the risk of that at a given place. And (c) how likely it is that other parties spam-block mail from them.

I'd look for TLS support for SMTP and IMAP; that may be the norm these days. The TLS situation for mail is a little unusual compared to most protocols, where on a new connection, some servers initially use the non-encrypted version and then upgrade via STARTTLS.

If you intend to leave your mail on their server rather than just using it as a temporary holding point until you fetch it, you might look into what their storage provided is.

I'd also see what the maximum size of any individual email that they permit is.