Ollama v0.32.6 Boosts Qwen3.5; KataGo, llama.cpp Get Performance & Stability Fixes

Ollama has released version 0.32.6, bringing notable performance improvements and API compatibility for local large language model inference. A key highlight is the enhanced speed for Qwen3.5 models running on Apple GPUs, achieved through automatic utilization of the model's MTP head for speculative decoding via the MLX engine. This optimization directly translates to quicker local inference on Apple Silicon devices, making Qwen3.5 a more efficient choice for users with M-series Macs. Beyond performance, this release also aligns Ollama's `/v1/chat/completions` streaming format more closely with OpenAI's wire protocol, specifically ensuring that the `role` field is present only on the first chunk of a streaming response. This change simplifies integration for developers and applications designed to interact with OpenAI's API, making it easier to switch between local Ollama instances and cloud-based OpenAI services without extensive code modifications. The update solidifies Ollama's position as a leading tool for accessible, high-performance local AI.

The popular Go AI engine, KataGo, has released version 1.17.2, a targeted bugfix update specifically addressing issues within its TensorRT backend. This release is crucial for users leveraging NVIDIA GPUs for accelerated inference, as it resolves several stability and correctness problems that could impact game analysis and self-play training. Given that KataGo relies heavily on efficient neural network evaluation (NNUE) for its strength, ensuring the integrity and performance of its GPU inference backend like TensorRT is paramount. This v1.17.2 update focuses exclusively on the TensorRT executables, indicating that the core model and other backends remain unchanged from the previous 1.17.1 release. For competitive Go players, researchers, and developers utilizing KataGo, upgrading to this version is highly recommended to maintain accurate and reliable high-performance GPU inference. The fixes contribute to the overall robustness of the engine, ensuring that search-vs-evaluation trade-offs and network architecture changes function as intended with optimal hardware acceleration.

llama.cpp, the highly popular C/C++ inference engine for LLMs, has released version b10291, addressing critical issues for users leveraging its Vulkan backend. The update primarily focuses on fixing problems related to Vulkan submission batching size and, importantly, introduces new debug tools designed to help diagnose the causes of `DeviceLost` driver errors. These `DeviceLost` errors can significantly disrupt local inference on consumer GPUs, leading to crashes or instability for users relying on Vulkan for accelerated operations. The addition of specific debug tooling (detailed in issue #26371) is a significant step forward for troubleshooting, providing more granular information about the root cause of these device failures. This release also includes fixes for submission threshold application. For the growing number of developers and enthusiasts running quantized models on consumer GPUs using llama.cpp with Vulkan, this update enhances stability and provides practical resources for resolving common hardware-related inference issues, making local LLM experimentation more reliable.