Why does this even need to be accounted for? People's outlooks vary widely.

For example, if you tell anyone with impostor syndrome who is an X that they have "X privilege" they're going to think, "Oh god, it's true, none of it was actually me, I'm just riding on X privilege, I'm such a phony".

Anyone who is a narcissist will be like, "Oh yeah, I'm an X and that privilege is so MINE!"

Anyone who loves cliques and is in one where "X privilege" is used a lot will be all, "Yes, I mean, totally, I know, I have X privilege too."

Anyone whose self-image is of being independent will react with, "Who are you calling an X? How dare you judge me by X's who I've never even met!"

Anyone with a really thick skin will just shrug it off: "Yup, if you say so. X privilege."

Anyone with a really thin skin won't be able to take it: "Nooo! Not me! I couldn't possibly have X privilege! Why are you so mean?!"

Anyone with sufficient exposure to social justice outlooks will have seen the whole argument, and will be offended or not based upon the refinement of the idea not how it naively presents.

And so on.

I'm happy to understand sources of variability when they matter crucially to an issue, but here I'm not sure there's much point. Simply noting that there are unfortunate connotations that have a reasonable chance of leading to what-is-claimed-to-be-a-misunderstanding (and do lead to exactly that misunderstanding) ought to be enough.