Unfortunately, much of your logic doesn't work here, because it's possible for a single term to have multiple meanings and uses.

Do you remember the word "retarded"? It was introduced as a neutral clinical term after words like "imbecile" took on too much of a negative character. And then "retarded" in turn was used as a slur, so it too was abandoned. We've even managed to make "special" into a slur in some contexts now!

Indeed, slurs can have pretty much any meaning, or no meaning at all. If someone shouts at me, "Get your ugly gebble ass off my street," gebble is most probably a slur. What other kind of word would go in that position save a slur? I am supposed to feel bad because I am a gebble, according to the shouter.

It doesn't matter if the word otherwise exists. You don't have to use it technically correctly. It's a slur if it's used to put people "of that type" down. (How bad of a slur it is depends on how much it is supposed to put them down.) That's the definition of "slur" in this meaning of the word--it's an aspersion. And saying someone has more of something can absolutely be an aspersion (c.f. "elitist").

So your conclusion may or may not be correct, but if it is correct it is not for the reasons you say, because those reasons are based on a misunderstanding of how slurs work.

You do have something of a defense of it not being used as a slur in practice (independent of the (invalid) argument that it isn't a slur because it's a description of a sociological reality). You could develop this more, but to be convincing you would need to consider more apparently problematic usages such as instances of the phrase "check your privilege". (Especially given the linguistic ambuity of the word "check" which can mean both "inspect" and "hold back".)

That there are much worse slurs does not need to be argued further, I think. That's pretty obvious! But if you're going to argue about whether "white privilege" can be used as a racial slur at all, at least make a good argument of it. (Personally I wouldn't have thought it worth the trouble. There are plenty of more intrinsically negative pairings of words like "white fragility" ("fragility" has a negative affect) that would deserve consideration first, if you wanted to explore this topic, no?)