Okay, but way less than 0.13% of people, say, die in airplane crashes, and yet we pay a lot of attention to make sure that number gets even smaller. Way fewer people than this have albinism--we find ways to allow them to live with a degree of safety and comfort; we don't just ignore them. You shouldn't just ignore something because it affects a small number of people if it affects them badly.

So basically sex is binary, but we're not so desperate that we can't recognize that the non-binary cases exist.