Sure. There are differences in musculature and such. But after a period of hormone therapy, those muscular differences disappear. When this is pointed out, transphobes inevitably move the goalposts and start talking about larger bone/frame size. Or they'll spread lies about how trans women must have larger frames and thus be a danger to their cis peers. Well here's this cis woman, with a massive frame size, who could utterly stomp all over the vast majority of trans female athletes. And she's allowed to play.

But people just make ignorant assumptions and assume that the athletic performance of trans women is anything remotely like the athletic performance of cis men. They start with that conclusion and work backwards from there.