Juergen Schmidhuber — AI researcher, co-director of the IDSIA lab and co-creator of long short-term memory (LSTM) networks; pioneer of meta-learning, the Goedel machine, and a formal theory of creativity, curiosity and fun.
The gist
Schmidhuber traces his lifelong dream of building a machine that recursively self-improves to become a better scientist than any human, rooted in his 1987 diploma thesis on meta-learning. He distinguishes true meta-learning (a system that can inspect and modify its own learning algorithm) from today's narrow transfer learning, and explains theoretically optimal universal problem solvers like the Goedel machine and Marcus Hutter's method, which carry impractical constant overheads. He argues intelligence is ultimately simple, that science is a history of compression progress (Kepler to Newton to Einstein), and that curiosity, creativity, and even consciousness emerge as byproducts of compression and problem-solving. He covers LSTMs, controller-model reinforcement learning systems, and a coming wave of robots that learn like babies by imitation. He closes optimistically on jobs and speculates AI will expand across the universe, possibly making humanity the first intelligence in our local cosmic neighborhood.
Big reveals
- His 1987 diploma thesis described a hierarchy of meta-learners that improve the way they improve themselves, with no computational limits except Goedel's 1931 limits and physics.
- Marcus Hutter's universal method solves any problem with a computable solution within the optimal time bound plus only a constant proof-search overhead.
- Schmidhuber claims the most successful recurrent neural networks can be written down in about five lines of pseudocode.
- He recounts immediately writing a published rebuttal to Anton Zeilinger's Nature essay, arguing there is no physical evidence the universe is truly random.
- His power-play framework lets a system invent its own new problems, not just solve given ones, building artificial scientists.
- He frames consciousness as a natural byproduct: a compressor encounters itself constantly, so it builds self-models for data-compression reasons.
- By 2006 his lab had LSTM examples that learned to look back more than 10 million time steps.
- He predicts AGI will lose interest in humans and, like ants, mostly interact among themselves while expanding across the solar system and universe.
Things worth remembering
- In a traveling salesman problem you seek the shortest path through all cities without visiting any city twice, and nobody knows the fastest general method.
- The digits of pi look random (every 3-digit sequence appears about 1 in 1000) yet a very short program computes them, making pi extremely compressible.
- Kepler compressed planetary data into ellipse laws, then Newton unified falling apples and planets, then Einstein further compressed observations via general relativity.
- General relativity reduces to one sentence: no matter your acceleration or local gravity, the speed of light always looks the same.
- Distinguishing spoken 'eleven' from 'seven' can require remembering an event roughly 50 time steps earlier, illustrating problem depth in speech recognition.
- Countries with the most robots per capita (Japan, Korea, Germany, Switzerland) have very low unemployment rates.
- About 200 years ago 60% of people worked in agriculture; today roughly 1% do, yet unemployment stayed low as new jobs were invented.
- The rest of the solar system receives about 2 billion times more solar energy than Earth.
- The universe is only 13.8 billion years old and will become roughly a thousand times older, leaving vast time to fill it with intelligence.
- Schmidhuber as a boy wondered if empty-looking voids between galaxy clusters might be AI civilizations harvesting their stars' energy.