I've come to the conclusion that "white fragility" can properly be defined as the colloquial name for the group-defensive reaction that white people have when their group is attacked (even if they don't explicitly recognize it). It's entirely plausible that there would be a set of responses that are somewhat characteristic of white people, given that there is some divergence in culture between races. There's actually a phenomenon there. We could name it.

But if we were to use this in this way, then we would also have "black fragility", "Democrat fragility", "Green Bay Packers fan fragility", "vegan fragility", and so on. Now we have a good way to say something disparaging-sounding about anyone when their group, or their status within their group, is attacked!

I don't think we need such a set of terms: I agree with your analysis of its downsides as applied to "white", and we hardly need to extend that negative stereotyping to other groups (even if there is some statistical truth behind the stereotype).

But I do think it can be said to refer to a real thing. You do arouse a somewhat characteristic set of responses when you tell someone they're an ape ("homo sapiens fragility") or that the Dodgers suck ("Dodgers fan fragility") or that pineapple does not belong on pizza ("Hawaiian pizza lover fragility").