Interview

Multi-GPU Kernels, Intelligence per Watt, Heterogeneous Inference, and More | YC Paper Club

At our latest YC Paper Club, researchers and builders presented on multi-GPU kernel optimization, intelligence per watt for local inference, AI-generated GPU kernels and benchmarking, heterogeneous inference infrastructure design, and GPU-accelerated game engines for reinforcement learning. Thanks to the following presenters:

Stuart Sul (Stanford / Cursor), John (Stanford), Mark (PyTorch / GPU Mode / CoreAuto), Misha (Marlo), and Brennan (Stanford)

Chapters:
0:00 – Francois Chaubard: The case for chip and kernel specialization
7:16 – Stuart Sul: Parallel Kittens - Systematic and Practical Simplification of Multi-GPU Al Kernels (https://arxiv.org/abs/2511.13940)
21:29 – Jon Saad-Falcon: Intelligence per Watt - Measuring the Intelligence Efficiency of Local and Cloud AI (https://arxiv.org/abs/2511.07885)
31:05 – Mark Saroufim: When Al Starts Writing Systems Code
47:04 – Misha Smelyanskiy: Why AI Inference Needs Heterogeneous Hardware
1:04:33 – Brennan Shacklett: Building a High-Throughput Game Engine that Runs ENTIRELY on the GPU (https://madrona-engine.github.io/shac...)
1:15:35 – Wrap-up & what's next

Source attribution: Original source

Interview details

Publication date: Aug 15, 2026
Duration: 1h 16m

Speakers

Brennan ShacklettPhD student, Stanford University Francois ChaubardPhD student, Stanford University Jon Saad-FalconPhD student, Stanford University Mark SaroufimCore Automation Misha SmelyanskiyCo-founder, Marlowe Stuart SulPhD student, Cursor

Viewpoints in this interview

Growing token demand drives specialization in AI chip architectures.

Growing token demand will drive extensive specialization in AI chips, with distinct architectures optimized for training versus inference data centers, as evidenced by early examples such as TPUv8 and hybrid NVIDIA/Cerebras deployments.

Francois Chaubard Chip Specialization Source at 0:08

ParallelKittens demonstrates efficient multi-GPU kernels with minimal code.

Writing efficient multi-GPU kernels requires navigating trade-offs in transfer mechanisms, computation-communication scheduling, and exposing performance-critical controls, as demonstrated by the ParallelKittens framework which achieves high performance with minimal code.

Stuart Sul Multi-GPU Kernels Source at 7:16

Local models route most inference on-device for energy and cost savings.

Recent improvements in local open-source models and consumer accelerators allow 80-90% of inference queries to be routed on-device, yielding significant energy and cost savings, though data center accelerators still lead in raw efficiency per watt.

Jon Saad-Falcon Local vs Cloud Inference Source at 21:28

Robust benchmarks are needed to evaluate AI-generated GPU kernels.

Evaluating AI-generated GPU kernels is challenging due to reward hacking and the need for robust benchmarks; competitive platforms like KernelBot and adversarial testing (e.g., KernelGuard) help identify cheaters but open problems remain in verification and compile-time optimization.

Mark Saroufim AI Kernel Evaluation Source at 31:05

Specialized systems for inference phases improve interactivity and TCO.

Inference is highly heterogeneous across phases, and co-designing specialized systems for prefill, decode, attention, and MoE kernels can improve interactivity and TCO, though full-stack integration with networking and data center design adds complexity.

Misha Smelyanskiy Heterogeneous Inference Infrastructure Source at 47:04

Batch simulators achieve 100x RL speedups but need simpler GPU programming.

Batch simulators that run thousands of game environments on a single GPU using entity-component-system patterns can achieve over 100x speedups for RL training, but require new high-level abstractions to simplify GPU programming for irregular workloads.

Brennan Shacklett GPU-Accelerated Simulation Source at 64:33