I know you mentioned that Russia has nuclear weapons--I was arguing with the emphasis, not the awareness.
France and the U.K. also have nuclear weapons. That means Russian leaders would have to do their own "nothing to lose" calculation, and unless clinically insane (at which point no actions that we take would relate sensibly to what they'd do) would recognize that invading the U.K. or France would likely result in annihilation-level responses.
Poland is a more difficult case, because it's part of NATO and therefore "has nuclear weapons" by proxy, but because it doesn't have its own nuclear weapons, one can conceive of a scenario where Russian leaders could think it worth risking attacking Poland.
Reasoning about geopolitical actions in the context of nuclear weapons is unlike reasoning about personal bullying at school, for instance. The bully's responses are limited by their capability: they cannot throw you through a wall, they cannot burn down the school with everyone inside, they cannot blow up the entire town. Bullies fundamentally are weak in that sense: fear is their greatest weapon, because they can't beat up everyone at the same time (in part because of lack of ability, in part because eventually teachers will intervene). If you look in scenarios where bullies actually can destroy everything (e.g. in gaming), standing up to bullies can instead just result in the bullies destroying everything. If you look in scenarios where people have huge ability to increase the level of violence, standing up to them often just provokes increased violence (Mexican drug cartels, for instance--you need to have a heck of a lot of power behind you before you can get Los Zetas to back down).
I don't see how sanctions in advance would have stopped Putin's actions in Ukraine. Sanctions work better as a deterrent than as a way to degrade a nation's ability to project force. If we'd used them more harshly years ago--well, then, yes, maybe the deterrence this time would have been greater. But as it was, scaling them up in response to aggression is the only way to utilize their deterrent effect, and because they're too slow and are hostile anyway, if anything, imposing them early would accelerate the likelihood and timetable for war.
I agree that Putin isn't (fully, anyway) in his right mind. But the timetable for action was so far in the past that except as a historical analysis (which is valuable), calling for decisive action now is unlikely to be helpful.
The biggest mistake, if you want to place one, was not in failing to deliver a more forceful response to Russian aggression. (Or a more forceful response to U.S. aggression.) Rather, the biggest mistake was the U.S. not rushing to embrace Russia after the end of the cold war, almost Marshall Plan style. Yes, yes, there were huge problems there (e.g. oligarchs), leadership inconsistent with our supposed values, and so on. We overlook that all the time, in much less weighty situations, based on interest (e.g. with Saudi Arabia). Russia as an uneasy ally would have had far more to lose when engaging in military actions against its neighbors.
At this point, as painful and tragic as it is, trying to intervene substantially more in Ukraine has a great risk of making things worse for everyone including Ukrainians. There is no "act decisively" any longer at the level of state actors. We messed up, and the time for strong decisive action is past (beyond the already semi-crippling level of sanctions).
The clearest non-disastrous way I see out is to do exactly what we're doing: arm the Ukrainians with plenty of weapons with high utility for defensive insurgence but low utility as part of an offensive campaign (thereby allowing Ukrainians greater power to stop Russian military advances, but not making them appear to be more of a threat), and allow Russia its "special operations" fantasy which admits the possibility of them in reality losing the war, but in rhetoric succeeding at the goals of the special operation, thereby allowing an end to the conflict.
But, as depressing as it is, the context now has to be one of not inducing overly fatalistic and defensive attitudes in someone who, as you put it, is "clearly isn't in his right mind". Because, nuclear weapons.