There are major sexual dimorphisms that are relevant for athletic performance in some sports and which cannot be reversed with hormones.

This is not true of race: there are not major racial dimorphisms that are relevant for athletic performance. (There are minor enrichments of sport-specific beneficial alleles in some ancestral groups.)

So, that is one very important reason that race and sex are different here. (It's not the only one.)

But come on--do you seriously not know this?