Anca Dragan: Human-Robot Interaction and Reward Engineering | Lex Fridman Podcast #81

Anca Dragan explains her work on human-robot interaction, where the robot's job is to optimize for what people actually want rather than what a programmer literally specified. She argues that humans who look irrational may simply be operating under different assumptions or simpler internal models, and shows how robots can use their own actions to gather information about human intent. The conversation covers inverse reinforcement learning, the difficulty of designing reward functions, autonomous driving as a game-theoretic problem with humans, and semi-autonomous driving's risks. It closes on mortality, the meaning of life, and how finiteness might belong in our reward functions.

• Anca reveals her husband proposed to her by building a seven-degree-of-freedom WALL-E robot that opened a Lego box.
  00:08:56
• Robots can deliberately nudge forward to probe a human driver's aggressiveness and update their model based on the reaction.
  00:25:43
• By modeling a human's intuitive (wrong) physics model, her team got people to actually land the Lunar Lander game.
  00:30:22
• She claims if you removed all humans from downtown San Francisco, autonomous driving would essentially be a solved problem.
  00:47:02
• She calls it irresponsible not to use lidar, while sympathizing with Musk's view of lidar as a crutch.
  00:49:37
• Lex pushes back on human-factors orthodoxy, saying drivers can be more energized as observers in some semi-autonomous setups.
  01:08:34
• The very state of the world (e.g. shoes lined up on the floor) leaks information about human preferences to a robot.
  01:25:10

• Inverse reinforcement learning infers what reward function a human's behavior is optimal with respect to.
  00:18:27
• Boltzmann rationality models humans as choosing options stochastically in proportion to their utility.
  00:19:59
• She frames human-robot interaction as an 'under-actuated' system where you influence but cannot directly control people.
  00:42:54
• Anca's hobby is watching hundreds of hours of pedestrian video to learn about human behavior.
  00:44:59
• 'Civil inattention': if you avoid eye contact while running, people move out of your way.
  00:45:29
• Goodhart's law: once a metric becomes a target, it stops being a good metric.
  01:14:48
• Robots should interpret specified rewards as evidence of intent, not as literal universal laws.
  01:23:06
• Anca says we love existing so much precisely because it ends, and that finiteness should inform reward functions.
  01:30:25

Books, products and media the guest or host genuinely endorsed here — with the buy link.

Affiliate link — we may earn a commission at no extra cost to you.