A developer preview of DirectML support for Intel® AI boost

With the release of DirectML 1.13.1 and the ONNX Runtime 1.17, we are excited to announce developer preview support for NPU acceleration in DirectML, the machine learning platform API for Windows. This developer preview enables support for a subset of models on new Windows 11 devices with Intel® Core™ Ultra processors with Intel® AI boost.

This initial release was developed using open source models and in close partnership with our launch partner, Samsung. “NPUs are emerging as a critical resource for broadly delivering efficient machine learning experiences to users, and Windows DirectML is one of the most efficient ways for Samsung’s developers to make those experiences for Windows,” said Hwang-Yoon Shim, VP and Head of New Computing H/W R&D Group at Samsung Electronics.

DirectML is a low-level API that provides a common abstraction layer for hardware vendors to expose their machine learning accelerators. DirectML works with any DirectX 12-compatible device including GPUs and now NPUs. Support for Intel® Core™ Ultra processors with Intel® AI Boost was developed in close collaboration with Intel®. You can read about the latest from Intel® in their latest technical blog: Scaling AI in Samsung PCs with Intel® AI Boost and Microsoft.

With this developer preview, DirectML targets the NPU on new Intel® Core™ Ultra devices and runs a subset of machine learning models that are optimized for NPU acceleration. NPU support in DirectML is a work in progress and we are eager to hear your feedback and learn from your experiences.

“We are excited to enable our developer community to harness the power of the industry’s first NPU with DirectML support on Intel® Core™ Ultra processors! These exciting AI features and capabilities are only possible thanks for our deep partnership with Microsoft’s Windows AI team, and we’re looking forward to continuing this partnership and realizing many more exciting AI experiences!”, said Michael Langan, Senior Director of Engineering & General Manager NPU, Intel®.

Getting started with NPU support in DirectML

To use NPU support in DirectML, you need the following:

• A Windows 11 device with an Intel® Core™ Ultra processor with Intel® AI boost.
• The DirectML 1.13.1 NuGet package, ONNX Runtime 1.17, and the latest NPU driver from Intel®.

Limitations of this developer preview

This developer preview is an early release that is intended to give you a glimpse of the future of DirectML with NPU support.

• NPU support in DirectML is only available for Intel® Core™ Ultra processers with Intel® AI Boost at this time.
• NPU support in DirectML is currently only compatible with a subset of machine learning models that have been targeted for support. Some models may not run at all or may have high latency or low accuracy. We are working to improve the compatibility and performance of more models in the future.
• Models that are functional and were not selected for this developer preview may have some stability and reliability issues. We are continuing to expand coverage for additional models suitable for NPU.

How to provide feedback and report issues

We appreciate your feedback and would like to learn from your experiences with NPU support in DirectML. To provide feedback and report issues, use the GitHub issues on the DirectML repository. Make sure to include the details of your device, your Windows build, your DirectML application, and your machine learning model when reporting an issue.

Thank you for your interest and participation in this developer preview. Stay tuned for future releases of DirectML with expanded NPU support.