You simply have the evolutionary pressure wrong.

Firstly, all else being equal, there is a positive selection for genes that encourage men to impregnate fertile women of any age without playing any role in childcare. It's basically zero-cost to the man, and even if the child's prospects are poor, some possibility of the genes being passed on are better than none. Except all else is not equal. Our societies allow us to cooperate for mutual benefit, and it is not mutually beneficial (or beneficial for a tribe) for this kind of thing to happen. But this doesn't mean that the original selective pressure is wrong--just that it's more subtle, in that the man has to not get clobbered by society.

There are lots of ways to do this: genuinely not feel attraction; genuinely feel attraction but suppress it fully (save subconsciously) by cultural training; genuinely feel attraction but pretend not to out of deference to cultural demands; genuinely feel attraction and act on it while trying not to get noticed; have different rules for your tribe and strangers, where you can through charm or violence try to have zero-responsibility impregnation, etc. etc..

The point is, this whole axis does not depend on a maximal probability of survival of the mother or child. Because men have the possibility of exploitative sexual behavior.

Therefore, you can't conclude that the evolutionary pressure is for men to avoid attraction to fertile females. Indeed, you'd assume the opposite, unless humans were very, very well controlled by society for a very long time (long enough so there was a huge negative pressure to even having attraction, as it would tend to result in death or banishment of the male, thereby drastically reducing his fitness and thereby the fitness of genes encouraging attraction). Giving the widespread existence of sexual violence during war, this seems extremely doubtful.

So it seems like yet another aspect of human behavior that we must use culture to overcome. That we use culture to make it worse, as you point out (e.g. hypersexualization), is not good.

(Note also that there is a fitness advantage for genes leading women to want to have sex relatively early, because human gestation is slow, and if you start early you might get an extra child (fitness advantage), and anyway if you reproduce in less time, you also get a fitness advantage...of course since humans are huge complicated creatues there are lots of disadvantages, too--the social support has to be there, for instance. But if conditions are right (e.g. adequate support from the tribe, old enough so chance of complications is low even if later it they would be lower yet, etc.), fitness is improved by relatively early reproduction. So we shouldn't be surprised to see it. Likewise, men should be interested in a long-term partner at about the same age that it makes sense for women to start having children for fitness reasons--and in historical societies if you do the calculation, I'm sure it's well under 18. What makes sense changes with society, though--the question is how completely society can drive our behavior and impulses.)