Responsibilities are burdens. The degree of burden should not be discounted.

"From each according to their ability" has no awareness of the burden this responsibility entails.

One of my points is that, in fact, rights feel free. But they aren't; they can only be attained by taking on some corresponding responsibility, and we need to consider the burden entailed by taking on that responsibility.

This doesn't mean that burdens need to be precisely equal, but we do need to balance them. And one of our responsibilities is to not be an undue burden on other people.

Thinking in terms of responsibilities does not free you from needing to very carefully choose what the responsibilities should be, just as thinking in terms of rights does not free you from needing to think very carefully about what should be a right. It's still a lot of hard work, and there are many complex issues on which reasonable people may disagree, and much discussion should be had.

But if you start with rights first, you're less likely to even be able to have the discussion. For instance, if you say that everyone should have a right to food and housing, and in contrast someone else says that everyone should have a right to their own possessions and the fruits of their own labor, it is not that obvious how to proceed. When cast as responsibilities, at least the distinction is thrown starkly into contrast: one position is that we have a responsibility to provide food and housing to those who don't have it; and the other position is that, no, we don't have any such responsibility.