If you use a morally-deaf value-only argument, quite possibly the slaveholder could have "more value to society" in that skillful direction of the effort of multiple people can make a bigger improvement in outcome than can the effort of a single person.

The argument against slavery is not that it is inefficient on the individual level (though it may be at times). It is that it is cruel. It is that if you include not just the production of cotton, or whatever, in the equation, but also the well-being of every human, slavery pushes the equation so far into the red that no skillful application of direction by a slaveowner can ever hope to make things positive again.