Statistics has always seemed to me stuck in a place that resembles physics before Newton. Lots of pieces that work on their own, but no unifying theory. This drives me crazy, and is probably a reason that there is so much hating on statistics. I discovered Kolmogorov Complexity reading a paper on vacation last week, and wonder if that could be used as a basis for a unified theory of statistics. Here’s a reasonable starting point:
Kolmogorov Complexity – A Primer
which leads to
Phlog 