Rex Kerr
1 min readMar 21, 2024

--

This isn't even correct. Correlation will extract a linear term of the mutual dependency between two variables when density is approximately constant over the observed range--it gives you a closed form answer for p(a, b). You can cast it as mutual information under those conditions if you want; but mostly people don't bother.

Furthermore, when density isn't approximately constant, the low mutual information overall may be misleading: of course if there isn't much variation you will only learn things in the rare cases where there is, but that rare variation may be highly informative. So you just trade one set of quantitative flaws for another.

The far deeper problem is assuming that either linear correlation or mutual information has something to do with how things work (i.e. causality). A greater use of causal inference modeling is advisable! Merely using mutual information instead of correlation mostly isn't. (Mutual information is a nice tool to add to one's statistical toolbox.)

Anyway, favoring correlation has little to do with whether something is evidence-based. It's a decent method with some sizable flaws which should be used with awareness of the flaws. So is mutual information.

--

--

Rex Kerr
Rex Kerr

Written by Rex Kerr

One who rejoices when everything is made as simple as possible, but no simpler. Sayer of things that may be wrong, but not so bad that they're not even wrong.

No responses yet