Architecting Future Intelligence.

The map for the autonomous stack series. What the parts of a self-driving system do, why driving is harder than it looks, what an operational design domain is, why latency and rare events dominate the engineering, and how the architecture has moved from separate modules toward learned end-to-end systems with safety checks around them.

Cameras, lidar, radar, ultrasonics and microphones each measure something different and fail in different conditions. What each one is good and bad at, why robotaxis carry all of them while some carmakers use cameras only, and why the point of combining them is that they fail differently.

Before sensors can be combined, the car must know exactly where each one is mounted, how each camera lens bends light, and when each measurement was taken. Why tiny errors become big ones at distance, how factory calibration and calibration trees work, how cars keep calibration correct while driving, and why time synchronization matters as much as position.

GPS alone puts a car somewhere within a few meters, which is the width of a lane. How self-driving cars get to centimeters by combining satellites, motion sensors, wheel odometry and map matching; why filters gave way to factor graphs; and what happens in tunnels, snow and other places where one source of position fails.

An HD map tells a self-driving car what the road looks like before its sensors can see it: exact lane geometry, stop lines, which lane leads where. What these maps contain, how they're built, why keeping them fresh is the hard part, and why the field is moving toward cars that draw much of the map themselves while they drive.

Perception turns millions of raw measurements per second into a short list of what's around the car, where it is and how it's moving. How fusion moved from combining boxes to combining features, why a bird's-eye view became the standard workspace, why occupancy matters for things that aren't boxes, and how perception copes with the open world.

A self-driving car has to guess what every car, cyclist and pedestrian nearby will do in the next several seconds, knowing there's usually more than one plausible answer. How prediction grew from physics to learned models that read the map, why it outputs several futures with probabilities, why predicting agents one at a time breaks down, and how the field measures success.

Planning turns everything the car knows into a decision: go now, wait, or creep forward. How planners balance safety, legality, comfort and progress; how classical search and optimization work; why learning to plan from human driving is harder than it looks; why a simple rule-based planner once beat the learned ones; and where hybrid planners stand now.

Reflections from building behavior prediction for autonomous vehicles: why prediction is a probability over futures, why the car's own actions change what everyone else does, and why that feedback loop is the real reason prediction and planning are hard to separate.

The same denoising recipe that paints images can generate robot motions and driving trajectories. Why that handles 'two good ways to do it' better than ordinary regression, how Diffusion Policy works (action chunks, receding horizon), how it runs fast enough for control, and where it leads: flow-matching action experts in VLA models and diffusion planners for driving.

The core idea behind Stable Diffusion, FLUX, Veo and robot policies: destroy data with noise in small steps, then train a network to undo one step at a time. The forward process, what the network actually predicts, sampling, noise schedules, latent diffusion and flow matching, explained through one image.

The network inside the denoising loop. Why diffusion started with convolutional U-Nets, why it moved to Diffusion Transformers, and how a transformer is told the noise level, the prompt and where each patch sits. U-Net, DiT, adaLN-Zero, cross-attention versus MMDiT, and the prediction targets.

A tour of the video generators that defined the field and what each one revealed: Sora's spacetime patches, Movie Gen's fully published recipe, Veo's native audio, and open models like HunyuanVideo, CogVideoX, Wan and Open-Sora that brought the recipe to anyone with GPUs.

Video models produce motion that looks right far more often than motion that is physically right. What the research shows about how much physics they actually learn, the four ways people are adding it (physics losses, simulators in the loop, post-training on simulated data, and world foundation models), and how physical plausibility is measured.

How an image or video diffusion model goes from raw web data to a product: curating and re-captioning data, pre-training from low to high resolution, weighting noise levels, then quality tuning, preference tuning and reinforcement learning. Plus the training-health signals worth watching.

A trained diffusion model is a direction-finder, not an image generator. Samplers decide how to walk from noise to an image, and guidance decides how hard to steer toward the prompt. DDPM, DDIM, ODE solvers, classifier and classifier-free guidance, negative prompts, and the knobs that trade speed, variety and prompt-following.

A video model is an image model that also has to keep things consistent and moving plausibly over time, at many times the cost. How video diffusion grew from image models with temporal layers to full space-time transformers, why video VAEs compress time, how long videos are made, and how video quality is measured.

A diffusion model paints by taking dozens of small steps, each a full pass through a large network. Distillation trains a student to get there in one to four. Why steps are the cost that matters, and the three families of tricks (follow the path, match the result, add a critic) behind SDXL-Turbo, FLUX.1 schnell, DMD2 and real-time video.

When you type a prompt into an image or video generator, the model is only one stage. A map of what surrounds it: input safety checks, prompt rewriting and expansion, generation, output safety checks and watermarking, and the newer agentic pipelines that plan shots and call several models. Built from Google's documentation and open-source releases from Google, Alibaba and Tencent.

Why a large language model can check several guessed tokens for about the cost of generating one, how the acceptance rule keeps the output exactly the same, the main ways to make guesses (draft models, n-grams, Medusa, EAGLE, multi-token prediction), and when it stops helping, including for vision-language and mixture-of-experts models.

A map of the models that see, read and act, for robots and for cars: where their training data comes from and how it's curated (with Qwen-Drive's pipeline as a worked example), the three ways to turn a language model into a controller, how they're trained, and why evaluating them is harder than building them.

One support team, one screenshot, one model. How a Qwen-style vision-language model reads an image, how a mixture-of-experts layer routes each token to a few experts, how the router and experts are trained without collapsing, and what changes when you deploy it.

How models learned to look at a picture and talk about it, told as a short sequence of ideas: CLIP's shared space, BLIP's captions, Flamingo's frozen language model, LLaVA's thin adapter, and natively multimodal models.

Reinforcement learning is how an agent learns what to do now when the reward comes later. The core ideas (states, rewards, value, Q-learning and deep Q-networks), built up on one tiny grid world.

Backpropagation answers one question: when a network's prediction is wrong, how much should each weight change? Here's the idea, the three equations that do the work, and a tiny network you can trace by hand.

A short map of the Python tools that show up again and again in coding interviews (heaps, memoization, union-find, tries, binary search) and the gotchas that cost people time, organized by the shape of the problem.

Retrieval-augmented generation lets a model answer from your documents instead of its memory. A map of what it takes to make it work beyond a demo: hybrid search, reranking, chunking, video, freshness and how to measure it.

A map of how vision-language models, image generators and video generators are built: the data, the two kinds of visual tokens, the training stages and how they're evaluated, with links out for depth.

AI can now write a clean, tested change in seconds. This post is about the part it can't see: who owns the code, who depends on it, and how to stay the person who understands the system.

Serving a model that answers questions is one problem. Serving one that queries databases, runs code and sends messages is another. A map of what changes: the agent loop, tools through MCP, sandboxes, safety and versioning.

On a modern GPU, arithmetic is cheap and moving data is expensive. A map of why that's true, how fusion and tiling fix it, how FlashAttention works, and when it's worth writing your own kernel.

A model that doesn't fit on one GPU has to be split across many. A map of where training memory goes, how DDP, ZeRO and FSDP divide it, when to add tensor and pipeline parallelism, and how big runs survive failures.

A guide to this site's MLOps series: eight short modules that follow a model from the files its data lives in to the GPUs that serve it, with the one trade-off each layer is about.

Where the time goes when you send a prompt to a model in production: the network, the queue, tokenization, batching, the two very different GPU phases, and streaming the answer back. Following one request from start to finish.

Airflow, Kubeflow, Temporal and Ray all 'orchestrate,' but they sit at different layers and solve different problems. A map of the layers, using one daily retraining pipeline to show which tool owns which part.

Pre-training gives a model knowledge; post-training gives it behavior. A map of the main steps (supervised fine-tuning, LoRA, preference tuning, and reinforcement learning with checkable rewards), following one model as it becomes a SQL assistant.

How data is laid out on disk and in memory decides how fast everything above it can go. A map of Parquet (for storing) and Arrow (for computing), why ML pipelines use both, and the file-layout mistakes that quietly slow training down.

When you serve a large language model, the GPU usually runs out of memory for conversations long before it runs out of compute. A map of the KV cache, PagedAttention, continuous batching and the other tricks that let one GPU serve many more people.

Self-driving stacks were built as a chain of specialist modules. Foundation models promise one model that sees, reasons and acts. Here's what that looks like, why teams keep a fast path and a safety layer around it, and what's still unsolved.

GPU utilization is a lagging indicator — the real battle is in the data pipeline. A practitioner's deep dive into PyTorch DataLoader internals, zero-copy data pumps, WebDataset streaming, and the exact questions this gets you in ML system design interviews.

A deep dive into XGBoost — how second-order Taylor approximations and sophisticated regularization make it the dominant algorithm for structured data, bridging mathematical rigor with system engineering excellence.

Part 4 of a comprehensive guide to agentic AI design patterns. Covers common failure modes, safety mechanisms, verifiable pipelines, and how to build reliable production systems.

Part 3 of a comprehensive guide to agentic AI design patterns. Covers specialized patterns: embodied agents, 3D scene understanding, imagination loops, multi-agent societies, error recovery, and self-debugging.

Part 5 of a comprehensive guide to agentic AI design patterns. Covers 2025 trends, cost optimization, case studies, production checklist, and the state of the field.

Part 1 of a comprehensive guide to agentic AI design patterns. Covers the fundamentals: ReAct loops, planning, tool use, self-consistency, and graph-based reasoning.

An intuitive introduction to the Transformer architecture — from the attention mechanism to self-attention and cross-attention, using language translation as a concrete example.

An intuitive introduction to Variational Autoencoders — how compressing data into probabilistic codes enables machines to generate realistic images, sounds, and structures.

Part 2 of a comprehensive guide to agentic AI design patterns. Covers production-ready patterns: memory management, supervisor/orchestrator, parallel tool execution, and hidden reasoning.

An exploration of modern agent systems, with math, analogies, and examples. From ReAct loops to multi-agent societies, discover the design patterns that make AI agents think, act, and fix themselves.

My research journey from wireless communication foundations to solving the camera calibration bottleneck that enables autonomous vehicle vision.

How we used deep learning to automatically calibrate traffic cameras by observing vehicle motion—work that won Best Paper Award at ACM BuildSys 2017.