Why, in a deep sense? I don't know. Maybe because concerns about it are treated by some with indifference at best and contempt at worst.
Why in a shallow sense? Because false accusations are possible and scary. It doesn't help that the possibility of being raped is also scary. It's not like the two cancel each other out.
Which men? Well, Henry Cavill for one: https://www.yahoo.com/news/henry-cavill-says-hes-scared-date-post-metoo-doesnt-want-called-rapist-alone-155417524.html
Men in the workplace: https://www.forbes.com/sites/ellevate/2019/01/09/dear-sir-dont-let-metoo-make-you-afraid-of-me/?sh=109cb52f3b59
Men going on dates: https://www.marketwatch.com/story/young-single-americans-are-tired-confused-and-scared-about-dating-during-metoo-2018-02-27
(And MGTOWs too, but most of them have serious issues they need to sort out before they should date anyone, so I'll count that one as a win.)
It doesn't seem to be at severely pathological levels yet, but there's enough even despite the abysmally low rate of prosecuting rape.
So I don't think there's a huge amount of leeway here to be drastically more cavalier about making mistakes with prosecution.