“AI Snake Oil” Chapter 3: “Why Can’t AI Predict the Future?” (Patreon)
Content
while this chapter didn’t feel like it contributed much to my understanding of AI (not sure if others felt similarly? lmk), a lot of points it made tied back to some of the ideas from my technocracy video via what is basically a breakdown of the limits of data science:
Techno-optimists have argued that statistical tools can replace the scientific method because of the sheer number of correlations they can find in data that no human scientists could conceive of.
So for context, this is how a lot of research in astronomy works these days. The time I started doing astronomy research was around the beginning of the “big data” era of astronomy, in which, by combining the increasingly massive astronomical surveys that were being developed in the field with new techniques in machine learning, there came to be a whole category of astronomy whose research boiled down to “run this data set through this model, see what correlations it spits out, and try to come up with a physical explanation for them.” This was very normalized to me, and I don’t want to say that no interesting results came out of this kind of research, but until reading this I hadn’t thought of it in terms of “replacing the scientific method”, even though it kind of is. Another kind of unfalsifiable astrophysics, if you will, starting with the extremely open ended question of “what’s interesting about this data?” rather than any particular hypothesis.
Of course, while this can work for a lot of astrophysical contexts where the behavior of the system in question is governed by a handful of deterministic physical laws, it doesn’t translate to all contexts:
In the 1950’s MIT professor Jay Forrester pioneered an approach called system dynamics, which tried to extend the use of simulations to social systems. He tried to use simulation to model a whole city, with an eye to solving thorny social problems like urban blight. Here, things went badly wrong. And we don’t have to speculate too much to find out why—a city is far from a physical system.
I feel like this is the technocratic impulse in a nutshell: X method works for stars/particles/whatever, so let’s apply it to human beings—inevitably to unsuccessful ends. Again tying back to my technocracy video, it makes me think of Howard Scott and his insistence that society is metrical, but it also makes me think of something interesting that I saw happen during the early days of covid. Basically a bunch of the astronomers who did this kind of “apply machine learning to big dataset” work started trying to do epidemiology research, regardless of whether they actually had any domain knowledge of epidemiology. From my understanding, most of this research was, at best, pretty much useless—kinda a flimsy gesture of ‘look I’m helping’ from people whose work otherwise has decidedly zero impact on society—and at worst, actively harmful—see, the physicists at University of Illinois who made their campus’s covid re-opening plan, which ended up being a massive failure. I think the examples discussed in this chapter did a good job of illustrating the importance of domain knowledge in the understanding and possible applying of algorithms to whatever context (i.e., the specific reasons predictive AI would fail in a given situation), and at the same time, the ignorance of some data/computer scientists to that reality.
In a similar vein, this bit was kinda funny to me:
Before Matt presents the results from the Fragile Families Challenge in academic talks, he usually asks the audience to guess how the teams performed. Some of the most optimistic audience members are computer scientists and data scientists, who have seen how well AI can perform in other domains. When confronted with the disappointing result, this is also the group with the most follow-up questions and ideas for improving predictions. One of the most common questions: Are samples from four thousand families enough?
Like, of course the only possible problem in the eyes of the data-pilled crowd is just that they need more data. I guess that does kind of tie a little bit to the ideology driving our current AI moment. Like, yeah gen AI or whatever isn’t perfect, but if we just steal MORE data and have MORE computing power and MORE advanced algorithms, we WILL eventually achieve AGI—rather than concede any inherent limitations. Interesting to read this particular explanation to why that kind of reasoning is flawed.
next week's reading is finally getting to gen AI, verrrry curious to see what we get there!