Small Models Get Smarter, Llama.cpp Keeps Shipping, and On-Device AI Finds New Form Factors

Tools

SmolLM3 Pushes Sub-1B Model Quality to New Limits

HuggingFace's SmolLM3 shows that small models are closing the gap on much larger ones for common tasks, which is a big deal for anyone running inference on edge hardware. If you're deploying locally and haven't revisited small model quality lately, this is worth a fresh look.

llama.cpp Continues Its Aggressive Release Cadence

The llama.cpp project keeps shipping incremental but meaningful updates that improve performance across a wide range of hardware configs. For builders running local inference, tracking these releases is basically required maintenance.

Ollama's Official Docker Image Makes Local Model Deployment More Reproducible

Ollama landing as an official Docker image removes a real friction point for teams that want consistent local model environments across dev and prod. This is the kind of infrastructure boring that actually matters.

MLC-LLM Expands Hardware Support for On-Device Inference

MLC-LLM continues broadening its hardware target list, which matters as on-device AI moves beyond phones and laptops into more constrained environments. If your deployment target is anything other than a cloud GPU, this project deserves attention.

News

Mistral Medium 3 Raises the Open-Weight Mid-Tier Bar

Mistral's Medium 3 targets the practical middle ground where most production workloads actually live, not the benchmark-chasing top end. The price-to-performance story here is compelling for anyone building on open weights at scale.

Analysis

New Research Benchmarks Efficiency Tradeoffs in Open-Source LLM Inference

This paper gives a clearer picture of where the real performance bottlenecks are in open-source inference stacks, which is useful if you're trying to decide where to optimize. Less speculation, more empirical data on what actually moves the needle.