Why are you talking about the most extreme opposition? This is roughly as useful as someone on the right saying, (1) a loud and overly influential subset of the the left endorses "punch a Nazi", and (2) they also call us all Nazis. Every member of the right has their life constantly threatened by the left.

This is wrong not because there aren't a few crazies who both think you should punch any Nazi you can find, and that anyone remotely right-wing a Nazi. If that were true, then everyone on the right would be in grave danger. It's wrong because it's a minority view.

Overall, views aren't that extreme--not with who is a Nazi, or what the attitude is towards trans people. In the U.K. no group that they polled comes anywhere close to the characterization you give, even if some are pretty hostile: https://yougov.co.uk/topics/politics/articles-reports/2020/07/16/where-does-british-public-stand-transgender-rights

So it would seem that there's a pretty decent chance of building bridges, as long as you're not actively trying to burn them down.