Biology is messy. In that context, "binary" means the kind of separation you get between male and female sex. It's strongly differentially regulated with multiple mutually-inhibitory feedback pathways that drive development and ongoing maintenance towards one of two alternatives.

A spectrum is something like height, political orientation, body mass index, skin pigmentation, conscientiousness, hip to waist ratio, etc. etc.. It is not something like sex, the differences between hands and feet, fingernail and skin, and so on.

It's hard enough to understand what is going on and how things work without intentionally using language to confuse and confound issues.

If you want to say "there are cases that do not fall cleanly into a male/female dichotomy", sure, yes, biology is messy and that's true for basically any dichotomy you can think of: occasionally things happen differently, often because of some genetic difference but occasionally simply because the regulatory program isn't 99.99999999+% reliable. (That's the reliability you need to typically not find even one person on the planet who has a trait that doesn't fit whatever dichotomy you have in mind.)

But the dichotomy itself is very much there. Sex isn't a spectrum. It's binary, with variation, plus some rare large-outlier noise.