The price of writing software with AI just fell again

At a one million token context, DeepSeek V4 Pro uses roughly 27% of the inference FLOPs and 10% of the KV cache of its predecessor, V3.2. The smaller V4 Flash gets to around 10% of the FLOPs and 7% of the cache. I read those two lines in the paper three times. Swings that size don’t make an existing product a bit cheaper. They move whole categories of product from “not viable” to “obviously worth doing”.

There are two models in the family. V4 Pro has 1.6T parameters with 49B active. V4 Flash has 284B with 13B active. Both handle a 1M token context natively and both are open weights on Hugging Face. The parameter counts got the headlines. The efficiency is the story.

What follows is the parts of the paper I think are new, then what cheap long context means if you build with agents.

The architecture changes that matter

DeepSeek kept the bones of V3: DeepSeekMoE for the feed-forward layers, multi-token prediction, the usual Transformer skeleton. The changes are in a few specific places.

Hybrid attention

This is the big one. Vanilla attention is quadratic in sequence length, which is the structural reason long context is expensive. V4 splits attention into two regimes and mixes them across layers.

Compressed Sparse Attention (CSA) compresses the KV cache along the sequence and then runs sparse attention on top. It descends from the sparse attention work in V3.2, but the compression step cuts per-token state hard before sparsity even starts. Heavily Compressed Attention (HCA) goes the other way: it compresses more aggressively and keeps attention dense over the much smaller state. CSA layers do cheap long-range scanning. HCA layers keep dense behaviour where it’s needed.

A small sliding-window path runs alongside both for the most recent tokens, because recency is where dense attention earns its keep. You end up with a layered cache, and you can keep a million tokens around without paying a million tokens of compute per step.

Constrained hyper connections

The paper calls manifold constrained hyper connections (mHC) an upgrade to residual connections. Plain residuals add the previous activation back, which keeps information flowing but also lets representational drift compound over many layers. Hyper connections generalise residuals into several parallel paths. mHC constrains those paths to a manifold, which keeps the geometry well behaved as you stack hundreds of layers over trillions of training tokens. The practical result is more capacity to route information through depth, without the training instability you usually get when you start fiddling with the residual stream. The team says recomputation and fused kernels keep the wall-clock cost small.

Muon instead of Adam

V4 trains with Muon, which orthogonalises updates for matrix-valued parameters rather than treating each weight as a scalar. The claimed benefits are faster convergence and steadier training, with a hybrid ZeRO strategy to make its extra state fit at scale. It’s one line in the abstract. Optimiser choice rarely moves the needle, though, and a lab committing to a new one for a flagship run tells you it has stopped being a research curiosity.

Smaller routing changes

The routing affinity activation moves from sigmoid to sqrt(softplus), the limit on routing target nodes goes, and the dense feed-forward layers in the first few blocks become hash-routed MoE layers. Each is small. Together they add up, and hash routing the early layers is a trick I hadn’t seen in a flagship model before.

Half the paper is systems work

If you only read the architecture section you miss where a lot of the efficiency comes from.

The team wrote a single fused kernel, MegaMoE, that does dispatch, expert compute and combine in one pass, overlapping compute, communication and memory access. They validated it on both NVIDIA GPUs and Huawei Ascend NPUs and report 1.5x to 1.73x speedups for general inference, and up to 1.96x on latency-sensitive work like RL rollouts and fast agent serving. The CUDA version is open source as part of DeepGEMM. Fast agent serving is the workload most people building agents care about, so nearly 2x on that path is a big deal.

They also moved kernel development to TileLang, a language whose lowering rules line up with NVCC so the same source can compile to bit-identical CUDA when needed. It replaced what would have been hundreds of small PyTorch operators. Being able to prototype a kernel and check it against a hand-written reference in hours changes how fast a team can iterate.

The detail I liked most is batch-invariant, deterministic kernels. Most attention kernels split work across streaming multiprocessors in ways that aren’t bitwise reproducible across batch shapes. V4 uses one kernel that keeps a whole sequence on a single SM and a second for the partial tail that preserves the same accumulation order, and swaps cuBLAS for DeepGEMM end to end because cuBLAS isn’t batch invariant. The payoff is the same numbers in pre-training, post-training and inference. If you’ve ever chased a model that behaves differently in evaluation than in training, you’ll know what that’s worth. I’ve spent enough of my life reconciling payment ledgers to have a soft spot for anything that makes two systems agree to the last bit.

FP4 quantisation-aware training covers the MoE expert weights and part of the attention indexer. On today’s hardware the gain is mostly memory and bandwidth. The paper suggests future hardware could make the same path 1x to 3x faster, so it’s partly a bet on where silicon is heading.

Then there’s the unglamorous one that makes agent economics work: a KV cache that can live on disk. Shared prefixes (system prompts, tool definitions, conversation history) are stored so the server can pull them back without prefilling them again, with a different storage strategy for each attention regime.

If you run agents, the prefix is nearly everything. A long system prompt plus tool definitions, repeated on every turn, is the dominant input cost. Reusing it from disk is what makes cache-hit pricing possible, and DeepSeek priced accordingly.

Post-training

V4 post-trains in two stages. First, separate expert models for maths, coding, agents and instruction following, each trained with SFT and then GRPO reinforcement learning. Then those experts are distilled on-policy into one model, which learns as a student against reverse KL. It sidesteps the usual problem where multi-domain RL produces skills that fight each other inside one set of weights. It’s one of the cleaner uses of on-policy distillation I’ve seen in a flagship release.

What the pricing changes for builders

DeepSeek published prices alongside the paper. V4 Flash was $0.14 per million input tokens on a cache miss, $0.0028 on a cache hit and $0.28 per million output tokens. V4 Pro launched under a temporary discount at $0.435 input on a miss, $0.003625 on a hit and $0.87 output, with regular rates around four times higher. On output alone, that’s well over an order of magnitude below the list prices of the closed frontier models at the time.

Agents make the gap wider than the headline suggests. An agent that runs with a fixed system prompt, persistent tool definitions and a growing history gets most of its input from cache after the first turn, and on Flash cached input is close to free. Frameworks like Hermes and OpenClaw also produce a lot more output per step than chat does, because of reasoning traces and tool calls, so cheap output matters more for them than for a chatbot. Leaving an agent running all day stops being a budget conversation.

Pro is the interesting middle option. In thinking mode it scored close to the top closed models on many agent and coding benchmarks at a fraction of the price, which makes it a sensible default for harder multi-step work. Flash is for throughput and for running lots of small agents in parallel. Both take the full 1M context, which agent frameworks love for loading whole codebases or long histories.

My honest read is that the closed frontier models still earn their price on the hardest reasoning and on the long tail where reliability can’t slip. For a lot of everyday agent work, though, the floor dropped by an order of magnitude in one release.

So what’s expensive now?

This is the part I keep coming back to. If generating code costs almost nothing, the cost of a piece of software moves almost entirely into the things around the generation: checking it, deciding whether to trust it, shipping it, running it and being able to explain later what happened. Those were always expensive. They’re now most of the bill.

Cheaper models also mean more agents doing more work, which multiplies the amount of change a team has to review and stand behind. Getting faster at writing code doesn’t make you faster at shipping it unless the rest of the pipeline keeps up. That’s the problem we’re working on with Kizuna and Voxeltron: letting agents build under rules you set, and getting what they build running somewhere you control.

A few ideas from the paper I expect to become normal across the field. Hybrid attention, where different layers use different cache shapes. Constrained residual connections, once one frontier model has shown the training stability holds. Bitwise-reproducible kernels across training and inference, which nobody asks for until they’ve had them. And on-disk shared prefix caches, which anyone selling agent infrastructure will end up building.

The paper and checkpoints are at huggingface.co/collections/deepseek-ai/deepseek-v4. The MegaMoE kernel ships as part of DeepGEMM.