A Better Newspaper

Speculative decoding is an LLM inference acceleration technique in which a small "drafter" model proposes multiple tokens that a larger model verifies in a single forward pass, exploiting the memory-bound nature of autoregressive decoding. A cluster of recent research addresses how to tune and extend this technique for production serving. TableTune addresses the problem that the optimal draft length shrinks as batch size grows, and that vLLM's hand-written draft-length tables are not measured or corrected; it introduces a tuner that times decoding steps across batch sizes and draft lengths to compute throughput-maximizing tables in minutes (arXiv:2610.38968, October 30, 2026). Separately, research on sparse Mixture-of-Experts (MoE) models found that prior theory predicting a concave speedup-batch size curve (with gains peaking at intermediate batch sizes due to expert saturation) does not hold for recent fine-grained MoE models, which instead show higher speedup at low batch sizes across 10 models ranging from 30B to 1T parameters and 8 to 512 experts (arXiv:2610.38981, October 30, 2026). CopyLoop extends speculative-style acceleration to Looped Transformers, which gain capacity through repeated application of shared blocks but are expensive to decode at full recurrent depth; it introduces a training-free decoding algorithm combining low-cost context-copy proposals with shallow screening and continuous recurrent scheduling (arXiv:2610.38822, October 30, 2026). ## Significance As LLM serving costs remain a major constraint on AI deployment economics, incremental but compounding improvements to speculative decoding — spanning draft-length tuning, MoE-specific behavior, and new architectures like Looped Transformers — are directly relevant to inference cost structures for any enterprise deploying LLMs at scale.