Moms have the final say (barring accidents and non-consentual encounters) in whether to have a child, but fathers have the final say in whether they're going to bail on the child (and mother).

There is plenty of scathing criticism for such fathers (c.f. "deadbeat dad").

So I think the premise is wrong that the criticism is directed against single mothers and not absentee fathers. One could argue that the balance isn't quite right, but not that "deadbeat dads" get a free pass from society.