It is also possible for people to recognize that if people are acting unpleasantly, and happen to be members of some group--even if they are obnoxious while announcing that they're part of some group!--it's possible to blame the people, not the whole group.
This is kind of the whole idea of not being prejudiced, isn't it?
So I think that even if some trans people act antagonistically, even if some religious people act antagonistically, even if some people of one race act antagonistically against another race, still those of us who basically embrace the classical liberal value that people are first and foremost individuals, not representatives of whichever group we or they place them in, we can still accept people.
It might take a bit longer, however.