I agree that this is unworkable. And I find Harris' attempts to bolster his views rather embarrassing--half of it seem so boil down to how he modulates his voice while saying "but clearly there are some cases".
However, in addition to your analysis (which I pretty much agree with), I think there's a pretty good argument that not only can you get an ought from an is, you can't supportably conceive of "ought" any other way than as from an is: https://ichoran.medium.com/solved-the-meaning-of-life-and-the-is-ought-problem-b8a62283ba