Vladimir Vapnik — Co-inventor of support vector machines and VC theory, a foundational figure in statistical learning whose work has been cited over 200,000 times. He has worked at AT&T, NEC Labs, Facebook AI Research, and is a professor at Columbia University.
The gist
In this second conversation with Lex Fridman, Vladimir Vapnik distinguishes engineering intelligence (imitating human behavior) from understanding it (the science of intelligence). He builds on Plato's world of ideas and Vladimir Propp's 31 narrative units to argue that intelligence rests on discovering a small set of universal 'predicates' that constrain the admissible set of functions. He frames a challenge: achieve state-of-the-art MNIST handwritten digit recognition using roughly 100 times fewer examples by encoding good predicates like symmetry. The discussion ranges across weak versus strong convergence, deep learning's limits, music criticism as a source of predicates, and reflections on mortality and the meaning of life.
Big reveals
- Vapnik claims engineering intelligence (imitation) and understanding intelligence are 'completely different' problems with different goals.
- He issues his core challenge: beating MNIST records using roughly 100-200 times fewer training examples requires genuine intelligence.
- When pressed on whether one needs love and fear of death to recognize digits, he repeatedly redirects to digit recognition as the simplest path to intelligence.
- He dismisses deep learning's predicates and functions as mediocre, saying convolution is essentially a single predicate Jan LeCun found 25 years ago.
- He rejects symbolic AI and logic-based reasoning, arguing 'just logic is not enough' to discover good predicates.
- He admits he was 'stupid for 50 years' working only on strong convergence and missing the power of weak convergence.
- He recounts an experiment using poetic descriptions of digits as 'privileged information' that dramatically improved learning.
- He says he does not greatly fear death, only regrets not finishing his work connecting predicates across music, art, and vision.
Things worth remembering
- Vladimir Propp's 1928 'Morphology of the Folk Tale' identified 31 narrative units that recur across Russian fairy tales, movies, and theater.
- Vapnik draws a philosophical lineage from Plato to Hegel to Wigner, framing intelligence as projecting a small world of ideas onto reality.
- He invokes the 'duck test'—looks, swims, and quacks like a duck—as an everyday example of weak convergence reasoning.
- Theory says using all possible predicates from Hilbert Space would require no training data at all.
- The fundamental trade-off: the more good predicates you have, the less training data you need.
- Vapnik says language is too complex to tackle this century and learning should start from simple tasks like digit recognition.
- He cites the Strugatsky brothers' idea that humanity will split into two divergent types: common people and very smart people.
- He observes that many managers at big companies studied English language and literature because it teaches them to understand life.
- The episode closes with Vapnik's maxim: when solving a problem of interest, do not solve a more general problem as an intermediate step.