They also won’t ever want sex

This is not true. Historically, "Many castrati lived rather promiscuous lives. Because their unions could produce no embarrassing offspring to explain, women saw them not only as beautiful, ethereal celebrities of the opera stage, but as prime candidates for affairs."

their penises will never grow to adult size

I bottomed for a trans women with a highly atrophied penis and had a great time. Tbh I don't know why anyone cares about penis size.

their voices will always sound like that of a little boy.

Their larynx won't drop, but they won't sound like little boys. Only some features of adult male voices are caused by biological puberty—there are a lot of other vocal characteristics that distinguish adults from children and men from women. I've known trans men who hadn't gone on testosterone who still had masc-leaning voices. And adult women sound different from little girls even though their voices don't drop during puberty.

Now, maybe you’re lesbian or asexual or something, and you’re fine with all that, but I imagine some women would not be pleased.

I'm bisexual, and I'd be totally down to date a guy who looked like this. As pointed out above, castrati were highly sought-after by women in the past. Obviously contraceptives negate the appeal of their infertility, but still, they must have been otherwise sexually appealing enough for women to want to sleep with them in the first place. Sure, it would be a loss for those who only like super-manly men, but that seems like an acceptable trade-off to me.