Rajat Monga — Engineering director at Google leading the TensorFlow team; involved with Google Brain since its 2011 start with Jeff Dean.
The gist
Rajat Monga traces TensorFlow from its origins in Google Brain's DistBelief system through the pivotal 2015 decision to open-source it, which he and Lex frame as a seminal moment for the tech industry. He explains key design choices like the graph-based architecture for production deployment, the move toward eager execution in 2.0, and the organic adoption of Keras as the recommended high-level API. The conversation covers the explosive growth of the TensorFlow community (41 million downloads, 1,800 contributors) and the challenges of maintaining backward compatibility while innovating rapidly. Monga also discusses building and managing a cohesive engineering team, Google's hiring process, and his earlier work leading search ads. He closes with thoughts on the ad-vs-paid revenue models on the internet and advice for beginners getting started with machine learning.
Big reveals
- TensorFlow was open-sourced in November 2015, with work starting in summer 2014 and the open-source decision made by late 2014.
- The decision to open-source TensorFlow is described by Lex as one of the seminal moments in all of software engineering, with Jeff Dean a big proponent.
- Francois Chollet started Keras before joining Google, on top of Theano first, and built it on his nights and weekends before the team integrated it.
- Chollet came over for what was meant to be one quarter to integrate Keras into TensorFlow; that quarter became about two years.
- TensorFlow growth is tied to the growth of deep learning itself, not just being a good tool plus listening to the community and building transparency.
- PyTorch's research-first approach (eager-style execution) influenced TensorFlow, possibly accelerating eager execution arriving in 2.0.
- Google evaluates engineers on whether a 'superstar' helps or hurts the team; productivity is judged broadly across the team, not just individually.
Things worth remembering
- Early Google Brain scaling runs reached up to 10,000 machines, showing that scaling compute and data improved results.
- The two early wins for Google Brain were speech recognition and the famous 'cat paper' on images.
- TensorFlow has been downloaded around 41 million times.
- TensorFlow has roughly 50,000 commits, nearly 10,000 pull requests, and 1,800 contributors.
- Models like Inception and ResNet-50 that are years old are still used by tons of people who want stability over peak performance.
- Python's creator Guido van Rossum held the 'benevolent dictator for life' role, prompting discussion of whether TensorFlow needs a single decision-maker.
- By late 2014 the team was already running machine learning models on mobile phones using customized handcrafted code.
- TensorFlow 2.0 was released in alpha at the dev summit, with a target to ship in the next quarter.