Ollama v0.32.9 Enables NVIDIA Nemotron 3.5 Lightning for Local Inference

Ollama, the popular open-source tool for running large language models locally, has released version v0.32.9, prominently featuring support for NVIDIA's new Nemotron 3.5 Lightning. This 30B mixture-of-experts (MoE) model, with 3B active parameters, is specifically designed for agentic workloads and optimized for high-volume execution of tool calls, result validation, and subagent delegation. Its integration into Ollama significantly lowers the barrier for developers and researchers to experiment with this advanced open-weight model on consumer GPUs. Nemotron 3.5 Lightning, as an open model, aligns perfectly with the local AI ecosystem, enabling users to leverage its specialized task execution capabilities without relying on cloud APIs. This release empowers practitioners to build and deploy sophisticated, always-on AI agents directly on their local machines, benefiting from the model's efficiency and NVIDIA's optimizations. The Ollama update underscores the growing trend of making powerful, frontier open models accessible for local inference, catering to the increasing demand for privacy-preserving and cost-effective AI development.

The vLLM project, known for its high-throughput inference engine, has rolled out its v0.27.0 release, featuring a massive update with 561 commits from 242 contributors. A highlight of this release is the full-stack landing of Kimi K3 support, encompassing core model files, kernels, and Python bindings. This integration allows users to efficiently run the Kimi K3 model, an important addition to the growing landscape of open-weight models for local and accelerated inference environments. Beyond Kimi K3, vLLM v0.27.0 introduces a plethora of general improvements to its architecture, enhancing performance and stability for a wide range of LLMs. The substantial number of contributions reflects the active development and community engagement around vLLM, continuously pushing the boundaries of inference acceleration through techniques like PagedAttention, KV cache optimizations, and efficient batching. This release solidifies vLLM's position as a go-to framework for demanding local inference workloads, especially beneficial for users with consumer GPUs aiming for high throughput.

Ollama's v0.32.7 release introduces initial support for Meta's Muse Glimmer model, a new open-source, agentic, and multimodal model. Notably, this initial implementation leverages Ollama's MLX engine, providing optimized performance specifically for Apple Silicon. This makes Muse Glimmer readily accessible for a significant segment of users equipped with consumer-grade Apple hardware, allowing them to run advanced multimodal AI agents directly on their Macs. Muse Glimmer represents a push towards more capable local agents, capable of answering users and calling tools within a single generation. The focus on Apple Silicon for initial support highlights a commitment to broadening the accessibility of cutting-edge models for local inference, aligning with the blog's focus on consumer GPU compatibility. While future updates are expected to expand support and optimizations to NVIDIA, AMD, and other platforms, this release provides a crucial entry point for developers interested in exploring Muse Glimmer's multimodal agentic capabilities on a popular local platform.