Francois Chollet — Deep learning engineer and AI researcher at Google, creator of the Keras library and author of the paper 'On the Measure of Intelligence' and the ARC challenge.
The gist
Lex Fridman talks with Francois Chollet about how to define and measure general intelligence. Chollet frames intelligence as the efficiency with which a system acquires new skills on tasks it was not prepared for, drawing a sharp distinction between intelligence (a process) and skill (its output artifact). He critiques deep learning and GPT-3 as powerful pattern-matchers that lack true reasoning and out-of-distribution generalization, and explains the design philosophy behind his ARC challenge, which makes human cognitive priors explicit. The conversation ranges across language as an operating system for the mind, psychometrics and the g-factor, the Turing test, the Hutter compression prize, and ends on the meaning of life as contributing ripples to culture.
Big reveals
- Chollet disagrees with Chomsky, arguing language is not the foundation of cognition but an operating-system layer on top of it.
- He gives his core definition: intelligence is the efficiency with which you acquire new skills at tasks you did not prepare for.
- Chollet is not convinced GPT-3 actually learns new tasks from a few examples; he thinks it is mostly pattern-matching against training data.
- He predicts a 100-trillion-parameter GPT would not change the conversation, because the real bottleneck is training data, not model size.
- He claims L4 self-driving is probably achievable but L5 is probably not via pure learning, favoring explicit engineered models.
- He recounts the famous mental-rotation experiment: answer time grows linearly with rotation angle, like a turntable in the brain.
- Chollet states humans are nowhere near the optimal intelligence bound, even if a hard limit exists.
- He admits he once believed cognition is compression but no longer does, calling compression merely a tool for cognition.
Things worth remembering
- Roughly 8 percent of all currency in the world is physical money; the other 92 percent exists only digitally.
- Skill is the only thing you can objectively measure, but seeing skill in a human signals intelligence because they were not born with it.
- On a 150-million labeled-image model, smaller human-annotated data beat far larger noisy data and trained faster.
- GPT-3 has no knowledge of the coronavirus because the world moved on after its training data was collected.
- People good at math tend to be good at physics and even at writing English essays, the correlations behind the g-factor.
- The g-factor is like physical fitness: real and broad, but not universal, just as a fit person still cannot fly.
- Elizabeth Spelke's core knowledge theory lists four innate priors: objectness, agentness, geometry/topology, and numbers.
- ARC machine performance started at zero while humans found it easy, then reached about 20 percent state-of-the-art.
- Chollet argues speed reading does not work and brain bandwidth is near-optimal, making him skeptical of Neuralink.
- He says even your most personal thoughts use words and concepts you did not invent; we are made of culture.
Recommended in this episode
Books, products and media the guest or host genuinely endorsed here — with the buy link.
Affiliate link — we may earn a commission at no extra cost to you.
On Intelligence
Jeff Hawkins
“i read it around the same time is jeff hawkins on intelligence which is a classic and he has this vision of the mind as a multi-scale hierarchy” — Francois Chollet 00:06:23Find it on Amazon
On the Measure of Intelligence
Francois Chollet
“let's get into your paper on the measure of intelligence that uh did you put on 2019 yes okay yeah november” — Lex Fridman 00:23:29Find it on Amazon