#vLLM

'A concise technical comparison of vLLM, TensorRT-LLM, Hugging Face TGI v3 and LMDeploy, highlighting when to use each stack for production LLM inference based on throughput, latency and KV behavior.'

Cerebras released MiniMax-M2-REAP-162B-A10B, a REAP-pruned SMoE checkpoint that trims experts to reduce memory while keeping near-original performance on code, reasoning and agentic tool calls.

'A practical comparison of the top 6 LLM inference runtimes in 2025, highlighting design tradeoffs, KV cache strategies, performance profiles, and where each engine fits in production.'

'IBM launches Granite 4.0 Nano: eight compact models (350M and ~1B) in hybrid SSM and transformer variants, built for local and edge inference with enterprise-grade governance and open licensing.'

'MiniMax-M2 is a compact MoE released with open weights on Hugging Face, optimized for coding and agentic tool use with about 10B active parameters per token and claims of ~2x speed at ~8% of Claude Sonnet cost.'

kvcached provides a virtualized, elastic KV cache for LLM serving on shared GPUs, reducing memory waste and speeding activation across colocated models.

'Liquid AI released LFM2-8B-A1B, an 8.3B-parameter sparse Mixture-of-Experts designed for phones and edge devices that activates about 1.5B parameters per token to balance latency and capacity.'

'GLM-4.6 expands context to 200K tokens, reduces token consumption on real-world coding benchmarks, and offers open weights for local inference and research.'

Alibaba's Qwen team published FP8-quantized checkpoints for Qwen3-Next-80B-A3B (Instruct and Thinking), enabling more efficient long-context inference and practical deployment of an 80B/3B-active hybrid MoE on commodity GPUs

'MoonshotAI released checkpoint-engine, a middleware that updates model weights across thousands of GPUs in about 20 seconds, enabling fast RL and large-scale LLM serving with minimal downtime.'

'AU-Harness is an open-source, high-throughput evaluation toolkit for audio LLMs that expands task coverage to diarization and spoken reasoning while improving evaluation speed and scalability'

'BentoML launched llm-optimizer to automate benchmarking and tuning of self-hosted LLMs and published a browser-based LLM Performance Explorer with pre-computed results.'

'Meta AI and UCSD's DeepConf uses token-level confidence to reach 99.9% on AIME 2025 with GPT-OSS-120B while reducing generated tokens by up to 85%, delivering higher accuracy at far lower compute cost.'

DeepSeek researchers released nano-vLLM, a compact and efficient Python implementation of the vLLM engine that balances simplicity with performance for LLM inference.