I get that men are terrified about false rape accusations, even though there is an extremely low chance of it happening, they are afraid. When someone talks about sexual assault, in their mind they feel they need to defend themselves by bringing up false accusations because in their mind it will discourage this crime, bring light to real victims, and thus protect themselves. I'm not a man but this is what I've been told. Then you have victims, who simply want their rapist and other rapists to face justice. They often know others around them who have also been sexually assaulted. When they talk about sexual assault and someone else brings up false accusations, it feels like they don't actually care about the victims and just want to soapbox about false accusations, stealing the spotlight. They might even be a toxic part of the manosphere.

The nature of the crime means it is usually the victim and the perpetrator alone and not in public, and the victim is more likely to know who the assaulter is. It can be a family member, friend, partner, etc. There usually isn't much evidence to prove sexual assault. So we have a problem. As you said, you don't believe a hefty amount of these sexual assaults should have laws enforced on them. I don't have a solution, other than a suggestion that people should be more empathetic towards victims. And not ask questions like yours to victims.