Yoshua Bengio: Deep Learning | Lex Fridman Podcast #4

Yoshua Bengio discusses what current deep neural networks are missing, arguing that more depth or data is not enough and that the field needs new training objectives focused on causal explanations and active, agent-based learning. He explores disentangled representations, the need to separate not just variables but the mechanisms that relate them, and lessons that classical symbolic AI can still offer. Bengio shares a measured view on AI risk, dismissing existential-threat scenarios as very unlikely while stressing real short and medium-term dangers like bias, surveillance, autonomous weapons, and threats to democracy. He also talks about machine teaching, instilling moral values and emotions like anger at injustice into machines, and how he persevered through the AI winter by trusting his intuition.

• Bengio argues that simply scaling networks from a hundred layers to ten thousand will not solve deep learning's core problems.
  00:09:30
• State-of-the-art deep learning still fails to understand even simple synthetic grid worlds, needing millions of examples where humans need dozens.
  00:11:35
• He considers AI existential risk very unlikely and not a pressing concern, though worth academic study like a meteorite strike.
  00:23:04
• The real near-term AI dangers he flags are surveillance, killer robots, job-market disruption, concentration of power, and discrimination.
  00:22:34
• Bengio wants to instill moral values into computers, starting with detecting unfair situations that trigger anger.
  00:29:54
• He says Ex Machina paints a totally wrong picture of how science works, which is collaborative and incremental, not one lone genius.
  00:24:38
• Bengio claims seminal events like AlphaGo are overrated because science really moves by small accumulating steps.
  00:38:13

• Biological brains can do credit assignment across very long time spans, something artificial neural nets struggle to do.
  00:00:30
• Bengio suggests efficient forgetting, remembering only important things, is a key part of human learning.
  00:03:39
• He notes purely unsupervised learning fails to produce high-level representations as powerful as those from supervised learning.
  00:06:17
• Current neural nets suffer from catastrophic forgetting, where learning new things can destroy previously learned knowledge.
  00:17:21
• Bengio uses a science fiction novel on an alien planet as an analogy for generalizing to new distributions via shared causal laws.
  00:19:56
• His group runs a project called BabyAI, a game with a teaching agent and a learning agent to study machine teaching.
  00:32:29
• Bengio's mother tongue is French, and he believes passing the Turing test is independent of which language is used.
  00:35:36
• He fell in love with AI through reading science fiction and getting hooked on programming one of the first personal computers.
  00:41:21

Books, products and media the guest or host genuinely endorsed here — with the buy link.

Affiliate link — we may earn a commission at no extra cost to you.