Good analysis. I don't know what he's on about with that stuff.
I am professionally embarrassed. (Not that he's had much research output in the last few years.)
(Incidentally, the reality paper math isn't just eye candy: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7517005/. But though they might be mathematically correct (I haven't checked, just verified that they're plausible), the assumptions are far too questionable to conclude very much under most circumstances. In particular, representational capacity and trainability aren't really considered, nor is a time-varying world. So it seems a better model of the tuning of biochemistry than of, say, neural activity.)