While I agree with your logic, I also think it's important--from the perspective of human wellness given the large number of correlates of well-being with IQ--whether there is a substantial or an irrelevantly small correlation of IQ with any particular trait. If the difference is substantial, we might want to take compensatory measures of some sort.

The best research regarding whether race has a bearing on IQ--for black people in the United States specifically--is, in my opinion, this paper: https://pubmed.ncbi.nlm.nih.gov/32955555/.

The bottom line is, albeit by mathematically trying to control for different conditions, unlike the military base study which sets all the conditions the same, is "if there is a difference, it's very small". The size of the study prevents determining changes less than about 2 IQ points, but the data is consistent with anything from about -2.25 to +0.15 IQ points (see supplementary eTable 3). The best guess is one IQ point, and...this encapsulates not just genetics but any unquantifiable effects of racism!

So all the effects of race and all the effects of ongoing racism, in terms of IQ, are...practically nothing. What matters to IQ is having educated, engaged parents who read to you, a mother who had folate in hre blood during gestation an dwho breastfed you after you were born, that parent(s) had enough money, etc.. The thing that is so wonderful about the paper is that it quantifies how important each of these things seem to be (on average --the study isn't large enough to look at individual differences but those are always super-important too: what works on average isn't the same as the best for each individual!) so we can think about trying to intervene in some of them.

(For an example that the right intervention can work miracles, my favorite example is the Harlem Children's Zone.)

So, anyway, I agree that it doesn't matter--if we find some group that happens to have lower IQ whether for an interesting reason or not (definitely true with some subsets of neurodivergent people; other subsets have higher IQ), we should focus on fixing things.

But in the case of race specifically, the answer seems to be: and after you fix things, any residual left is far far too small to notice if it's even there at all (which it might not be). So that provides even more impetus to fix things.