I don't understand why, though. How is this any more cogent than saying that numbers could truly be merely a question of quantity, because quantity has to be of a thing and numbers are precisely what abstracts that away?

People have to adjudicate between their own specific values all the time. For instance, you can specifically enjoy watching Star Wars, and you can also specifically enjoy going rock climbing, and you can also specifically enjoy having a conversation with your grandparents. But you can't do all these things at once, so the specific value you assign to each has, in fact, to be internally compared to decide which has the greatest overall value right then. These things involve different rewards and costs, too, but you still have to decide how to act.

So I don't think it's even true that all those specific utilities are qualitatively different and cannot be put on a quantitative footing, or at least establish an order on them--maybe a time-varying order, depending on how thirsty you are vs. how hungry, in the case of water and bread, but an order nonetheless.

So if you can decide whether to eat or drink; if you can decide whether to watch Star Wars, climb rocks, or talk with grandparents; then I think it is demonstrably the case that your central criticism is not a fatal flaw in the conception of the problem but rather a modest additional complication that needs to be addressed.