This is true to an extent, just like the same people defend freedom of speech while saying such awful things that one wants to smack them. But that doesn't mean that colorblindness itself is wrong, or that freedom of speech is wrong, just that people who cause harm are also people and can appreciate or appropriate things that are worthwhile.