skill2026-06-27
Below the Ice — Two Models Are Faster Than One: Speculative Decoding
Your language model writes one word at a time, each token waiting on the last — which is exactly why it feels slow. Tonight we go under the hood of speculative decoding, the draft-and-verify trick that DeepSeek's new DSpark paper uses to speed things up. A small, cheap model guesses several tokens ahead; the big, expensive model checks them all in a single pass, keeps the guesses it would have made anyway, and corrects the rest — so the output is identical to running the big model alone, just quicker. We build it from first principles, explain why verifying many tokens can cost about the same as generating one, why this matters now that inference cost and latency rule the economics of running models, what's overhyped about the headline speedup numbers, and the three things to watch next.