Thought-terminating cliches are used in all sorts of contexts outside of discussions of race and racial inequity. They're prevalent in political discussions, climate change discussions, discussions about Covid, you name it.
Also, while it is a problem, I think elevating these things above "micro" to full-fledged "verbal violence" is highly insensitive to people who have lost loved ones to actual physical violence, or who are now suffering with permanent disability and/or psychological trauma as a result of physical violence.
I could, for instance, claim that you are engaged in a genocide of nuance in language, but what does that accomplish except to diminish our reactions to claims of actual genocide? Wouldn't it be better if I saved strident language for truly dire situations, and instead said something like "you are promoting a regrettable diminution of nuance in language"?
Intellectually evasive techniques like you describe are bad enough if stated plainly, without diminishing worse things by putting "agree to disagree" on the same level as being lynched. In fact, a good number of these things (e.g. "agree to disagree", "now is not the time") are used as ways to try to disengage to avoid further verbal tension, let alone violence.
This isn't always appropriate--sometimes you just have to tackle the hard issues, because not doing so leaves lingering issues to fester and continues causing harm (worse harm than the social awkwardness of a difficult discussion). But let's try to keep the severity of different things in perspective, hm?
You can die a death from a thousand cuts, but it is not the case that each cut disembowels you.