Ollama Goes Multi-GPU as Apple's Quiet AI Stack Advances

Release

Ollama 0.9.0 Drops: Multi-GPU, Windows NVIDIA Support & a Local API

Ollama's latest update is a major leap for local AI. It now natively supports NVIDIA GPUs on Windows, opens up API access to other applications, and lays the groundwork for running models on multiple devices. This is a huge step for builders wanting to move beyond just Apple Silicon.

Research

Apple's MLX Research Pushes Efficient Model Training

Apple is quietly building a serious local AI stack. Their latest research on efficient model training and sparse tensor cores shows a focus on making models run better on consumer hardware, much of which is open-source on their MLX framework. This is the kind of foundational work that powers the next generation of on-device AI.

Tools

Exo: Cluster Your Devices for Distributed Local Inference

Exo lets you turn a bunch of your old phones and laptops into a single, unified AI cluster. It's a clever way to get cheap, private inference power by pooling existing devices. For builders with spare hardware, this is a practical path to running larger models without a cloud bill.

AutoRAG: Automated Pipeline for RAG Systems

AutoRAG automates the painful process of building a Retrieval-Augmented Generation pipeline. It handles the tuning and evaluation of different retriever and generator setups for you. This is a massive time-saver for builders who need to get a solid RAG system running fast.