Chris Lattner — Creator of the LLVM compiler infrastructure and the Swift programming language; senior director at Google working on TPUs, TensorFlow, and ML accelerators; former Apple developer tools lead and former VP of Autopilot software at Tesla.
The gist
Chris Lattner explains what compilers are and how they bridge human-written code and diverse hardware through front-end, optimizer, and back-end phases. He recounts building LLVM as a university project that grew into shared open-source infrastructure used by competitors like Apple, Google, Nvidia, Intel, and AMD, and later creating the Clang C/C++ compiler and the Swift language at Apple. The conversation covers compiler optimization, register allocation, the impact of Java and JavaScript, automatic differentiation, and Google's work on TPUs, bfloat16, TensorFlow, Swift for TensorFlow, and the MLIR project. Lattner also reflects on his time leading Tesla Autopilot's transition to an in-house vision stack and on working with Elon Musk. He closes by discussing work ethic and the origin of the LLVM dragon logo.
Big reveals
- LLVM began as Lattner's University of Illinois project with his advisor and a few research students, then matured at Apple.
- Google effectively owns Clang now because it cares so much about C++ and invests heavily in that ecosystem.
- Lattner started both Clang and Swift quietly on nights and weekends without telling anyone before they became official Apple projects.
- Doing a new language was 'heretical' at Apple because the team believed the iPhone's success was tied to Objective-C, which they loved.
- Memory safety became the core justification for Swift: you can't fix Objective-C's pointer safety without fundamentally changing the language.
- Lattner spent only five months at Tesla as VP of Autopilot software, leading the transition from a third-party to an in-house vision stack.
- At Tesla he saw the highest degree of employee turnover he'd ever witnessed at a company.
- The LLVM dragon logo's real origin is the classic 'dragon book' on compiler design, not the lofty traits Lattner publicly cites.
Things worth remembering
- LLVM standardizes the middle (optimizer) and back-end (codegen) so languages like Swift, Julia, Rust, and Clang share the same infrastructure.
- The C++ specification is roughly 1,400 pages, making it one of the most complicated programming languages.
- Compilers lower code from abstract syntax trees into a control flow graph, an assembly-level, mostly language-independent representation.
- LLVM has around 150 optimization 'passes' that code flows through, organized in complex ways affecting performance and compile time.
- Java in the mid-90s mainstreamed JIT compilation, garbage collection, and memory-safe portable bytecode that could run in the browser.
- TPUs use bfloat16, a 16-bit float format with a smaller mantissa and larger exponent, trading precision for range useful in ML.
- A multiplier's hardware cost is N-squared in mantissa bits but only linear in exponent size, making bfloat16 much cheaper to implement.
- Google's third-generation TPUs reach 100 petaflops in large liquid-cooled boxes.
- Swift can be statically compiled, dynamically compiled, and even interpreted, JIT-compiling statements line-by-line in a Colab/Jupyter notebook.
- MLIR was announced three weeks before this recording at a compilers-for-machine-learning conference as common infrastructure for ML compiler stacks.