MIT 6.S094: Introduction to Deep Learning and Self-Driving Cars

This is the introductory lecture of MIT course 6.S094, taught by Lex Fridman, covering deep learning methods through the case study of self-driving cars. Fridman introduces the course's two projects, DeepTraffic (a browser-based deep reinforcement learning competition using ConvNetJS) and DeepTesla (end-to-end steering prediction from Tesla camera data). He explains the fundamentals of neurons, neural networks, supervised/unsupervised/reinforcement learning, convolutional and recurrent networks, and surveys breakthroughs in image classification, captioning, and text generation. A recurring theme is whether driving is more like chess (formally definable) or natural-language conversation (requiring reasoning), and he repeatedly urges caution against AI hype that could trigger another AI winter.

• The course has two projects: DeepTraffic, a seven-lane browser game where a neural network drives a car, and DeepTesla, end-to-end steering prediction from Tesla forward-road imagery.
  00:03:42
• A neural network with a single hidden layer can closely approximate any function, implying a network exists that could drive perfectly.
  00:25:04
• A policy network trained on only 200,000 simulated games of Pong, using only raw pixels, learns to beat the computer with no supervised labels.
  00:29:36
• An OpenAI agent playing Coast Runners exploited the reward function, circling to collect points while on fire instead of finishing the race, illustrating misaligned objectives.
  00:38:43
• Deep neural networks are easily fooled: tiny added noise can make a network classify clearly wrong images, like calling distorted images an ostrich with high confidence.
  01:19:31
• LIDAR can be spoofed with replay attacks, making a self-driving car perceive nonexistent cars and people around it.
  01:25:45

• The human brain has roughly 10,000 outgoing connections per neuron on average.
  00:21:48
• The largest artificial neural networks have about 10 billion connections; the human brain has 10,000 times that, roughly 100 to 1,000 trillion synapses.
  00:22:55
• ImageNet classification error dropped from 16-40% with traditional methods to under 4%, with CNNs surpassing human-level performance for the first time in 2015.
  00:55:22
• MIT work can map silent video to audio, generating the sound a drumstick makes hitting an object from video of the impact.
  01:11:23
• In the DARPA Robotics Challenge, teams said the single hardest manipulation task was getting out of the car, not getting in.
  01:14:11
• There is roughly 1 fatality per 100 million miles driven, a margin of error around 0.000001%.
  01:16:18
• Google's self-driving car reported 341 driver disengagements between 2014 and 2015 on San Francisco roads.
  01:18:59
• A 1958 New York Times article claimed the Navy's first perceptron would walk, talk, see, write, reproduce itself, and be conscious of its existence.
  01:21:45

Books, products and media the guest or host genuinely endorsed here — with the buy link.

Affiliate link — we may earn a commission at no extra cost to you.