This seems technically true in at least some instances (I'm not sure we have the data for "hundreds of thousands", but post ice-age might be a supportable claim), but it's not a very relevant observation given that hunter-gatherer societies were outcompeted massively by agricultural societies, and the factors giving rise to the disparity have only been increasing over the past six thousand years.