If you transitioned to become a man, wouldn't you want to be called just that, "man"?

Either way, this "people with penises" is the type of waking in eggshells language that makes everyone annoyed just to satisfy a few perpetually slighted. It's perfectly fine to say "men", everyone knows what we mean, nobody is excluding anyone and you can't talk in ways that includes everyone because anytime you try, you'll just end up with yet someone else who feels insulted because you didn't include his particular interpretation of how he/she/xe/sho/is/zil/etc feels....

At one point we just need to remember nthst people typically aren't trying to insult anyone in normal speech and er shouldn't require and force people to our special and particular speech only or feel insulted.