But the brain clearly is regulated in some different ways than the rest of the body (there are lots of brain-specific transcription factors and receptors, for instance), so if we don't at least allow for the distinction, we're ignoring biology.

Additionally, nobody really cares if someone has "male feet" on an otherwise female body. I mean, it's maybe hard to find shoes and the individual might get teased in school, but once we grow up we decide it's pretty much irrelevant. So a large portion of "not doing it with other body parts" may well just be that we don't care about the other body parts as much.

(We do care about reproductive body parts a lot, and there is some rare variation there, but because the reproductive body parts produce hormones that cause a lot of the sexual dimorphism in body form, it's less frequent that there's a decoupling.)

So I don't think this is a compelling counterargument against the hypothetical argument-form that I raised.