State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490

In this multi-guest roundtable, Lex Fridman talks with researchers Sebastian Raschka and Nathan Lambert about the state of AI roughly a year after the 'DeepSeek moment.' They debate who is winning between US and Chinese labs, dissect the explosion of open-weight models, and explain the technical lineage from GPT-2 to today's architectures (Mixture of Experts, attention variants, KV cache). A large portion covers training stages, scaling laws, and the rise of reinforcement learning with verifiable rewards (RLVR) and inference-time scaling. They also discuss education and learning, careers in AI, tool use, continual learning, robotics, timelines to AGI/ASI, the business and consolidation landscape, NVIDIA's moat, and Nathan's ADAM Project to build American open models.

• Raschka argues there will be no winner-take-all in AI because researchers rotate between labs, so ideas are not proprietary; the differentiating factor will be budget and hardware, not technology access.
  00:03:09
• DeepSeek is described as losing its crown as China's preeminent open model maker as Z.ai (GLM), MiniMax, and Kimi Moonshot rise, with DeepSeek having kicked off a broad Chinese open-weight movement.
  00:05:48
• Despite rapid AI progress, the autoregressive transformer derived from GPT-2 remains essentially the same architecture; advances come from data, systems, and training stages rather than fundamental architecture changes.
  00:47:48
• Lambert reveals he was on the Ai2 team (Tulu 3 work) that coined the term RLVR before DeepSeek, while DeepSeek did the actual breakthrough of scaling reinforcement learning.
  01:38:29
• Unlike RLVR and inference-time scaling, RLHF has no clean scaling law where log-increasing compute yields linear performance gains; the seminal RLHF scaling paper is about reward model over-optimization.
  01:56:37
• Lambert details the ADAM Project (American Truly Open Models) to build and host high-quality US open-weight models to compete with China's open-source ecosystem, noting Ai2 received a $100M NSF grant over four years.
  03:49:29
• Cursor reportedly updates its Composer model weights every 90 minutes based on real-world user feedback, described as the closest thing to real-world RL happening on a deployed model.
  03:39:17
• Anthropic lost a court case in 2025 and owed $1.5 billion to authors, tied to torrenting books, even though buying and scanning books was cleared as legal.
  01:16:47

• DeepSeek's pre-training reportedly cost about $5 million at cloud market rates, and OLMo 3 cost roughly $2 million to rent the cluster, while serving millions of users costs billions in compute.
  00:52:01
• Leaders cite scaling laws holding for about 13 orders of magnitude of compute as a reason it is unlikely to stop soon.
  00:54:35
• Pre-training dataset sizes are measured in trillions of tokens; researcher models use 5-10 trillion, Qwen is documented up to 50 trillion, and closed labs are rumored to reach 100 trillion tokens.
  01:08:48
• A survey of about 791 professional developers found both junior and senior devs ship AI-generated code, with senior developers more likely to ship code that is over 50% AI-generated, and about 80% finding work more enjoyable with AI.
  01:31:09
• The guests note a brief roughly 10-year window where exams could be digital yet uncheatable; after AI, education is reverting to blue books and oral exams.
  02:11:51
• The '996' work culture (9am to 9pm, six days a week, about 72 hours) originated in China and has been adopted in Silicon Valley AI companies.
  02:21:07
• OpenAI's average employee compensation is cited as over a million dollars in stock per year.
  02:15:46
• Manus.ai, a Singapore-based company, was founded about eight months prior and had a roughly $2 billion exit; Groq (~$20B) and Scale AI (~$30B) are cited as consolidation deals.
  03:38:16
• In the DeepSeek R1 paper, the longer the model trains, the longer its responses grow, and the 'aha moment' is when the model self-corrects and recognizes its own mistakes.
  01:42:11
• NVIDIA's real moat is described as the CUDA ecosystem built over roughly two decades, not the GPU chip itself, though LLMs may make replicating something like CUDA easier.
  04:00:42

Books, products and media the guest or host genuinely endorsed here — with the buy link.

Affiliate link — we may earn a commission at no extra cost to you.