Reinforcement Learning — From Intuition to Algorithms
Advanced Fundamentals 35 min
A narrative-first walkthrough of reinforcement learning, starting with everyday intuition and ending with the math behind Q-learning and DQN.
Building intelligent systems with clarity.
I'm Gopi Krishna Tummala. I bridge the gap between Research Papers and Production Systems. Here is my blueprint for modern AI engineering.
Always learning
Choose Your Path
Structured learning paths organized by domain—from generative AI and production systems to autonomous vehicles and agentic intelligence.
🧠
GenAI Systems
From Transformers to Diffusion Models. Understanding the architectures and algorithms powering modern generative AI.
Explore
⚙️
MLOps & Production
Scaling, serving, and optimizing AI systems. Custom kernels, inference engines, and production infrastructure.
Explore
🏎️
Autonomous Systems
How self-driving cars actually work. Prediction, calibration, sensing, and closed-loop reasoning.
Explore
🤖
Agentic AI
From ReAct loops to multi-agent systems. Building intelligent agents that reason, plan, and act autonomously.
Explore
Journey at a glance
A quick tour through the roles, research labs, and collaborations that shaped my path in AI and autonomous systems.
-
2024 — PresentSenior Machine Learning Engineer
Adobe · Creative Cloud & Firefly · San Jose, CA
Leading large-scale data pipelines, training infrastructure, and responsible generative AI initiatives that power Firefly and Creative Cloud surfaces.
-
2022 — 2024Software Engineer, Prediction
Autonomous Vehicle Systems · Bay Area, CA
Shipped multi-agent prediction models for L3/L4/L5 autonomous vehicle fleets and co-designed the training framework and dataloaders that kept the stack fed with fresh data.
-
2018 — 2022Senior Systems Engineer
Qualcomm Research · San Diego, CA
Led prediction for Qualcomm's L3 highway autonomous driving stack—owning forecasting models, simulation harnesses, and post-drive analytics. Earlier built integration and test automation for the stack.
-
2018Graduated from The Ohio State University
Ph.D. Computer Science & Engineering
Dissertation on collaborative perception and behavior prediction for intelligent transportation systems.
-
2016Research Intern
Microsoft Research · Bangalore, India
Designed AutoCalib—large-scale traffic camera calibration with <10% speed error—in Microsoft's video analytics platform.
- 2013 — 2018
Graduate Research Assistant
The Ohio State University · Columbus, OH
Built SmartDashCam, Soft-Swipe, RoadView, and RoadMap; taught introductory programming; collaborated with Honda on live calibration and lane-level localization.
-
2012 — 2013Software Analyst
Standard Chartered Bank · Chennai, India
Developed reporting systems and automation scripts for global private banking infrastructure.
-
2011Project Intern
Tata Elxsi · Chennai, India
Optimized LTE PDCCH blind decoding algorithms and explored DSP-based radio prototyping.
-
2008 — 2012B.Tech Electrical Engineering & Minor in Mathematics
Indian Institute of Technology Madras · Chennai, India
Graduated with honors; led hostel council committees.
Featured work
A curated selection of recent publications and projects that explore robust perception, generative modeling, and multi-agent systems at scale.
-
-
The Custom Kernel Craze — Why Developers Are Taking the Wheel on GPU Optimization
Advanced MLOps & Production 25 minWhy modern AI teams are handcrafting GPU kernels—from FlashAttention to TPU Pallas code—and how smarter tooling is making silicon-level tuning accessible.
-
When AI Sees and Speaks — The Rise of Vision-Language Models
Intermediate GenAI Systems 22 minA high level view on how modern vision-language models connect pixels and prose, from CLIP and BLIP to Flamingo, MiniGPT-4, Kosmos, and Gemini.
-
vLLM and the Trilogy of Modern LLM Scaling
Advanced MLOps & Production 30 minHow PagedAttention, Continuous Batching, Speculative Decoding, and Quantization unlock lightning-fast, reliable large language model serving.
-
Diffusion — From Molecules to Machines
Intermediate GenAI Systems 28 minA clear introduction to diffusion and guided diffusion — how a simple physical process became a foundation for modern generative AI, from Stable Diffusion to robotics and protein design.
-
The Hidden Engine of AI — Training Frameworks and Resilience
Advanced MLOps & Production 30 minA reader-friendly guide to scaling AI models beyond the data pipeline—from training loops and distributed frameworks to checkpoints, mixed precision, and fault tolerance.
-
Module 02: How Cars Learn to See (Sensors)
Advanced Robotics 35 minFrom photons to decisions: How machines reconstruct 3D reality from 2D data. Covers cameras, IPM, radar, LiDAR, and sensor fusion in an intuitive, first-principles approach.
-
Module 03: The Bedrock (Calibration & Transforms)
Advanced Robotics 30 minIf you don't know where your eyes are relative to your feet, you trip. Covers intrinsics, extrinsics, SE(3) transforms, online vs. offline calibration, and time synchronization.
-
Module 01: The "Why" and The Architecture
Advanced Robotics 25 minWhy L5 autonomy is harder than a moon landing. Understanding ODD, latency loops, compute constraints, and the probability of failure in autonomous systems.
-
Module 04: Localization — The Art of Not Getting Lost
Advanced Robotics 25 minFrom GPS to centimeter accuracy: How autonomous vehicles know their exact position. Covers GNSS, IMU, wheel odometry, scan matching, and the Kalman Filter fusion that creates the "Blue Line."
-
Module 08: The Chess Master — The Art of Planning
Advanced Robotics 30 minFrom perception to action: How autonomous vehicles make decisions. Covers cost functions, game-theoretic planning, and the modular vs. end-to-end debate.
-
The Hidden Engine of AI — Datasets and Dataloaders
Advanced MLOps & Production 35 minA deep dive into how datasets and dataloaders power modern AI—from the quiet pipeline that feeds models to the sophisticated tools that make training efficient. Understanding the hidden engine that keeps AI systems running.
-
Diffusion for Action: Trajectories and Policy
Advanced Robotics 20 minHow diffusion models predict action sequences instead of pixels. Covers Diffusion Policy, world models for robotics, and connecting diffusion to reinforcement learning for autonomous systems.
-
Image Diffusion Models: From U-Net to DiT
Advanced GenAI Systems 15 minThe evolution of image diffusion models from U-Net architectures to Diffusion Transformers (DiT). Covers latent diffusion, the DiT revolution, and the complete image generation pipeline.
-
Modern Video Models & Motion: Sora, Veo 3, Open-Sora, and Motion Modeling
Advanced GenAI Systems 22 minDeep dive into state-of-the-art video generation models: Sora, Veo 3, and Open-Sora. Plus motion modeling techniques using optical flow, geometry, and diffusion fields.
-
Pre-Training & Post-Training: Building and Aligning Video Diffusion Models
Advanced GenAI Systems 20 minHow video diffusion models are built through pre-training and aligned through post-training. Covers the billion-frame training problem, DPO, RLHF, and the complete training pipeline.
-
Sampling & Guidance: The Dialects of Noise
Advanced GenAI Systems 18 minHow to accelerate diffusion sampling and control output quality. Covers DDIM, DPM-Solver, Classifier-Free Guidance (CFG), negative prompting, and inference optimization techniques.
-
Video Diffusion Fundamentals: The Temporal Challenge
Advanced GenAI Systems 18 minWhy video is harder than images, the DiT revolution for video, and how diffusion models learn temporal consistency. Covers V-DiT, AsymmDiT, and the mathematical foundations of video generation.
-
XGBoost — The Art of Boosting Trees with Second-Order Gradients
Advanced Fundamentals 25 minA deep dive into XGBoost — how second-order Taylor approximations and sophisticated regularization make it the dominant algorithm for structured data, bridging mathematical rigor with system engineering excellence.
-
Module 07: The Fortune Teller — The Evolution of Prediction
Advanced Robotics 35 minThe hardest problem in AV: predicting human irrationality. Covers the evolution from physics-based prediction to Generative AI, tracking the journey through Waymo Open Dataset Challenges.
-
Physics-Aware Video Diffusion Models: From Intuition to Research Frontier
Advanced GenAI Systems 35 minA deep dive into physics-aware video diffusion models: how researchers inject physical constraints into generative models, the three leading technical approaches, and their practical impact on robotics and scientific simulation.
-
Part 04: Agentic AI Design Patterns — Part 04: Failure Modes & Safety
Advanced Agentic AI 30 minPart 4 of a comprehensive guide to agentic AI design patterns. Covers common failure modes, safety mechanisms, verifiable pipelines, and how to build reliable production systems.
-
Part 03: Agentic AI Design Patterns — Part 03: Specialized Patterns
Advanced Agentic AI 50 minPart 3 of a comprehensive guide to agentic AI design patterns. Covers specialized patterns: embodied agents, 3D scene understanding, imagination loops, multi-agent societies, error recovery, and self-debugging.
-
Part 05: Agentic AI Design Patterns — Part 05: Production Guide
Advanced Agentic AI 35 minPart 5 of a comprehensive guide to agentic AI design patterns. Covers 2025 trends, cost optimization, case studies, production checklist, and the state of the field.
-
Part 01: Agentic AI Design Patterns — Part 01: Foundations
Advanced Agentic AI 45 minPart 1 of a comprehensive guide to agentic AI design patterns. Covers the fundamentals: ReAct loops, planning, tool use, self-consistency, and graph-based reasoning.
-
Part 02: Agentic AI Design Patterns — Part 02: Production Patterns
Advanced Agentic AI 40 minPart 2 of a comprehensive guide to agentic AI design patterns. Covers production-ready patterns: memory management, supervisor/orchestrator, parallel tool execution, and hidden reasoning.
-
Agentic AI: Machines That Finally Learned to Get Stuff Done
An exploration of modern agent systems, with math, analogies, and examples. From ReAct loops to multi-agent societies, discover the design patterns that make AI agents think, act, and fix themselves.
-
The Transformer — How Machines Pay Attention
Intermediate Fundamentals 20 minAn intuitive introduction to the Transformer architecture — from the attention mechanism to self-attention and cross-attention, using language translation as a concrete example.
-
Variational Autoencoders — From Compression to Creation
Intermediate Fundamentals 30 minAn intuitive introduction to Variational Autoencoders — how compressing data into probabilistic codes enables machines to generate realistic images, sounds, and structures.
-
The Role of Predictions in Closed-Loop Autonomous Driving
Advanced Robotics 20 minReflections on building production-grade behavior prediction systems for autonomous vehicles — and why closed-loop reasoning is the bridge between perception and planning.
-
AutoCalib – Automatic Traffic Camera Calibration at Scale
Advanced Robotics 15 minHow we used deep learning to automatically calibrate traffic cameras by observing vehicle motion—work that won Best Paper Award at ACM BuildSys 2017.
-
From Bits to Sight - A Journey from Wireless to Autonomous Vision
My research journey from wireless communication foundations to solving the camera calibration bottleneck that enables autonomous vehicle vision.
Recent writing
View all posts-
Tongue Twister Practice Sheet — Clarity and Fluency for Tech Talks
Beginner Fundamentals 5 minA structured articulation and pacing warm-up designed to help technologists speak with clarity and confidence in high-stakes meetings.
-
The 45-Minute Interview Flow: A Proven Framework for Cracking Google-Style Coding Interviews
Beginner Fundamentals 10 minA collaborative 45-minute thinking algorithm tuned for Google-style coding interviews—classify the problem, co-design an optimal approach, code with confidence, and handle follow-ups with ease.