Additional Contributors: Angela Serrano Brummett, Nick Eubanks, Pawan Tahiliani
With the release of Windows ML 2.9.42 Experimental, we are expanding on the Windows ML Runtime API, our new Windows-native inferencing path announced in October. This release is the first step in supporting a new MLIR-based backend, the DirectX Compute Graph Compiler (DX CGC) announced at GDC 2026, that we’ve designed and have been developing in conjunction with our partners at AMD, Intel, NVIDIA, and Qualcomm Technologies, Inc.

Included alongside this release are new docs and samples to guide you through using these new APIs. As you try them out, keep in mind this is an early release intended to get feedback on API shape and quality. As these APIs, the underlying compilers, and inference engine mature, we will add support for a broader range of workloads, with deeper performance optimizations, and broader silicon coverage.
Let us know your feedback and help influence our roadmap – we’d love to hear from you.
Check out our Ignite 2026 recording of BRK388: Build and deploy local AI at scale across enterprise PCs w/ Windows ML to hear more about these Windows ML enhancements.
New to Windows ML?
For those of you who are new to Windows ML, it is the unified, high-performance local AI inferencing framework for Windows. You can use it to run your own local AI workflows across Windows PCs spanning GPUs, NPUs, and CPUs. Local inference can help reduce latency, keep workload data on the device, and avoid per-token cloud inference charges.
Previously, Windows ML relied solely on ONNX Runtime (ORT) as its inferencing engine. Windows provides a system-wide copy of the runtime, along with hardware vendor execution providers (EPs) for hardware-specific acceleration, so you can keep your app size small.
In October we introduced an additional, experimental Windows-native inferencing path as new API surface for interacting with the underlying inference engine, and now we are expanding backend support to include DX CGC enabling MLIR-based compilation and workload execution.
Windows ML also includes the Windows ML CLI, an agent-ready command-line tool for getting your models ready to run with Windows ML. You can use it to convert, optimize, compile, and benchmark your models, and with this release we are adding support for converting your model to an MLIR-based representation – Compute Graph IR (CGIR) – for CGC based execution.

Windows ML Runtime API and DX CGC
The Windows ML Runtime API is the experimental Windows-native inferencing API for Windows ML. Powered by CGC, it gives you a single Windows-native execution path, with DirectX-native integration.
As these APIs mature for developers, this will mean:
- Hardware-specific acceleration, with fallbacks built in. CGC compiles your model for the target silicon using compiler backends from AMD, Intel NVIDIA, and Qualcomm Technologies, Inc. Built-in fallbacks keep models running and make the experience more consistent across hardware. Deep DirectX integration adds OS-level scheduling, resource management, and debuggability and profiling with PIX.
- A Windows-native programming model. The Runtime APIs give you one consistent set of APIs across models, hardware targets, and inferencing engines. Pass images, video, audio, and text to models directly through zero-copy paths, chain models into multi-model pipelines with a processor for each stage.
- Bring your own model, convert once, run on supported hardware. Convert your model once into a single portable representation, CGIR, that runs across supported hardware, without needing to maintain a separate model for each vendor. You can convert to CGIR with the Windows ML CLI, or Windows ML can do it for you at runtime for supported model formats. For common scenarios like text generation and speech recognition, the higher-level Task APIs build on the Runtime APIs and CGC, so you can run your model in just a few lines of code.

CGC is at the core of the new benefits that the Windows ML Runtime APIs provide. It is an MLIR-based compiler and inference engine for graph-based computation. It works from a portable representation of your model, the Compute Graph IR (CGIR), is framework and hardware agnostic.
CGC is optimized first for GPUs, with NPU support in progress. It progressively compiles the platform-agnostic CGIR of your model down to platform-specific code for the target silicon, applying general optimizations first, then device-specific ones, executes the result through Direct3D 12. The device-specific optimizations come from compiler backends built in partnership with AMD, Intel, NVIDIA, and Qualcomm Technologies, Inc.
Hardware vendor drivers that support CGC’s interfaces can participate in compilation for improved performance. Where they don’t yet, or for parts of a model that aren’t fully supported, workloads are executed through a built-in fallback path using shaders and other optimized kernels, providing a more consistent baseline experience across hardware.
This means a more cohesive and robust workflow for developers – bring your existing model, convert it to CGIR through the Windows ML CLI or during runtime, and get the benefits of both hardware-specific acceleration and shared fallbacks through one unified path.
For developers that prefer to stay on their existing ONNX Runtime-based path, that path remains supported and those ONNX workloads can also get some of the benefits of CGC through the Windows ML Compute Graph EP (CGEP), a new ONNX Runtime execution provider that runs the same ONNX models with no code changes.
For this initial release, both the Windows ML Runtime and ORT/CGEP paths support CGC for GPU workloads, with vendor-specific hardware acceleration for, with the NVIDIA GeForce 621.24 driver (or the equivalent 621.24 driver for RTX Spark). Vendor-specific acceleration for additional GPUs is planned to follow in future releases. Workloads targeting CPU and NPU are supported through the Runtime APIs via ORT-based execution. Supported model formats include ONNX, plus GGUF and Hugging Face SafeTensors through the Windows ML Task APIs for select workloads.
Explore: Recommended CGC Workloads | Runtime API Docs | Windows ML CGC Backend
What’s new with the Windows ML Task APIs
The Windows ML Task APIs are experimental, high-level APIs for common AI scenarios, including open-source and generative AI workloads. You supply the model, and Windows ML handles the rest. With this release, the Task APIs build on the Windows ML Runtime APIs and CGC, so they gain hardware-specific acceleration for:
- Text generation, for LLMs and small language models
- Automatic Speech Recognition (ASR), for example with and Nemotron 3.5
- Decision, for typed classification and routing with models like Laya (new)
Run open-source models directly . The Task APIs accept GGUF and Hugging Face SafeTensors model files (weights plus config) directly as inputs, alongside ONNX. You don’t need to convert or export the model first. Download a supported model from Hugging Face, and run it across supported silicon with the corresponding Task API.
Larger open-source models are now more practical to run locally, which unlocks scenarios like local coding. For example, you can run Qwen3.8-27B on your PC and use it as the model behind GitHub Copilot CLI through an OpenAI-compatible endpoint. You can find and fix bugs, run tests, and iterate on your code with the model running on your device and no per-token cloud charges.
For quick experimentation with Task APIs, start the local Windows ML server and point the OpenAI SDK you already use at it. If you already call models in the cloud, that’s a one-line change to the endpoint URL and key. When you’re ready for deeper integration, call the Task APIs in-process for more control and better performance. If you plan to deploy at scale, the Task APIs support opt-in watermarking of generated content.
As part of this release, you can run Hugging Face SafeTensors model files with the Runtime APIs and DX CGC across text generation, speech recognition, and decision, and GGUF files for text generation. For text generation, you can also load a GGUF file with the llama.cpp loader instead, with vendor-specific acceleration for GPUs via the CUDA GGML backend. See Windows ML docs for the full list of supported models here.
Try it: run a Hugging Face model locally through the Windows ML OpenAI-compatible server
# Prerequisites
# Download a Hugging Face model to a local folder
# hf download nvidia/Qwen3.8-27B-NVFP4 --local-dir C:\models\Qwen3.8-27B-NVFP4
# Start the local Windows ML server on the GPU with DX CGC:
# WinMLServer.exe C:\models\Qwen3.8-27B-NVFP4 --composition model --target gpu --context-tokens 32768 --port 8080
# Make sure you have installed the NVIDIA GeForce 621.24 driver and are running the workload on a supported NVIDIA GPU (see Windows ML docs for details)
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="<access key printed on startup>")
stream = client.chat.completions.create(
model="Qwen3.8-27B-NVFP4",
messages=[{"role": "user", "content": "Find the bug and fix it."}],
stream=True,
)
for chunk in stream:
print(chunk.choices[0].delta.content or "", end="", flush=True)
Explore: Generate Text API · Chat Completion API · Speech Recognition API · Decision API
Customer Momentum
Many app developers are already leveraging the local AI platform for Windows development – including Windows ML – to bring local AI to their apps. They’re building once for Windows PCs across AMD, Intel, NVIDIA, and Qualcomm Technologies, Inc, while keeping data on the device and avoiding cloud inference costs. We’re excited to see how the improvements in this release help them, and you, take local AI even further. Here’s what some of them have to say.

On a Windows PC, the Napster app can render an agent’s face right on the device, with local inference running on Windows ML with lower latency and no inferencing cost. With Windows ML, we can build AI experiences once and deploy them across Windows PCs powered by AMD, Intel, Nvidia and Qualcomm Technologies, Inc. Our engineers used the Windows ML CLI to measure performance for ONNX GenAI models and evaluate model tasks – in minutes instead of days.
– Ziv Navoth, CPO @ Napster
For the architects, engineers, life sciences teams and law firms that run on Egnyte, the most valuable files are often the ones they can’t send to a cloud AI service: an unreleased building design, a patient’s case report form, a privileged contract. Windows ML changes that. It lets Egnyte run AI models on the PCs and workstations our customers already own. A project team can compare two revisions of a drawing, or transcribe a project meeting, and the file never leaves the device. Firms get AI on their most sensitive work without giving up control of where their data goes, and saving a result back to Egnyte still needs a person’s approval.
– Suhail Pothigara, Head of Product, AI @ Egnyte
At 12x AI, we care deeply about making wellness support available to everyone while protecting their privacy. Copilot+ PCs make our vision possible on devices people already use every day. Using Foundry Local, Windows ML, and Aion Instruct, we built Echo Pro as an AI Wellness voice agent that runs offline. Employees get instant, judgment-free support anywhere, organizations safeguard their corporate information, and we drastically reduce recurring token costs.
– Greg Fields, CEO @ 12x AI
Get started
You don’t have to choose up front. These Windows ML APIs ship side by side, so start with the path that fits:
- Want to build on the Windows-native stack or try CGC on GPU? Reach for the new Windows ML Runtime APIs, backed by CGC.
- Experimenting with open-source models locally? Start with the new Windows ML Task APIs and pass in a Hugging Face SafeTensors model or a GGUF file.
- Already using ONNX Runtime and want to try CGC with minimal changes? Add the new Windows ML CG EP to your existing code.
- Shipping to production today? Use the generally available Windows ML ONNX Runtime APIs and hardware vendor execution providers.
To try the experimental APIs:
- Install Windows ML 2.9.42 Experimental, or the windowsml9.42a0 wheels for Python attached with the release on Github, then set up your project.
- For vendor-specific acceleration on NVIDIA GPUs, install the NVIDIA GeForce 621.24 driver (or the equivalent 24 driver for RTX Spark).
- Start with the recommended models for Windows ML and CGC. They cover LLMs such as Llama, Qwen, Gemma, and Phi; speech recognition with Whisper and Nemotron; decision with Laya; and non-LLM models such as YOLO, CLIP, and ResNet-50.
- Try the Runtime and Task API samples.
- Install the latest Windows ML CLI v.0.5 to convert, optimize, and benchmark your models, including conversion to CGIR.
Share your feedback on the Windows ML GitHub.
What’s next
This is just the first step, and upcoming releases will include expanded workload coverage, performance improvements, and broader silicon support.
Have feedback, hit a rough edge, or have an idea for what should come next? Search or file issues on the Windows ML GitHub. We’d love to hear from you, and this early feedback genuinely shapes the roadmap.
Missed the Ignite 2026 breakout session on Windows ML? Catch the BRK388 recording on the Ignite session page.
References
For more information, check out:
0 comments
Be the first to start the discussion.