…the proposal of this paper that we must refocus, working towards developing a framework for building systems that can routinely acquire, represent, and manipulate abstract knowledge, using that knowledge in the service of building, updating, and reasoning over complex, internal models of the external world.
If OpenAI tends to slant towards throwing more TPUs at the problem, Gary Marcus is on the other end of the spectrum.
In some sense what I will be counseling is a return to three concerns of classical artificial intelligence—knowledge, internal models, and reasoning—but with the hope of addressing them in new ways, with a modern palette of techniques.
While I certainly don’t possess expertise at the level of either of these parties, it seems fairly obvious to me that developing a causal model is a necessary prerequisite to more general purpose intelligence. Judea Pearl’s work on this topic resonates. In this paper, Marcus presents a bunch of great examples where the modern state-of-the-art fails to produce sensible outputs directly as a result to this lack of causal reasoning.