DirectML ❤ Stable Diffusion

Jacques van Rhyn

Text-to-image models are amazing tools that can transform natural language into stunning images. Stable Diffusion is particularly interesting. The base model can create images from text and since it’s open-source developers can customize it for their own needs and preferences. With some extra training, developers can fine-tune their model to generate images of any domain, subject, or style they want.

We are demonstrating what can be done with Stable Diffusion models in two of our Build sessions: Shaping the future of work with AI and Deliver AI-powered experiences across cloud and edge, with Windows.

We’ve optimized DirectML to accelerate transformer and diffusion models, like Stable Diffusion, so that they run even better across the Windows hardware ecosystem. Our goal is to enable developers to infuse apps with AI hardware acceleration at scale. For more on how Stable Diffusion lights up on our partners’ hardware with DML, check out:

 

Getting the best performance with DirectML

We worked closely with the Olive team to build a powerful optimization tool that leverages DirectML to produce models that are optimized to run across the Windows ecosystem. For more on Olive with DirectML, check out our post, Optimize DirectML performance with Olive

You can use Olive to ensure your Stable Diffusion model works as well as possible with DirectML. Make sure your model is in the ONNX format; you can use Olive to do this conversion. Once you’ve done this, follow the steps in our DML and Olive blog post

See here for a sample that shows how to optimize a Stable Diffusion model. We’ve tested this with CompVis/stable-diffusion-v1-4 and runwayml/stable-diffusion-v1-5. Stable Diffusion models with different checkpoints and/or weights but the same architecture and layers as these models will work well with Olive.

 

DirectML in action

Check out tomorrow’s Build Breakout Session to see Stable Diffusion in action: Deliver AI-powered experiences across cloud and edge, with Windows

See here for a Python sample showing how to use Stable Diffusion with Olive.

We also built some samples to show how you can use DirectML in general in C++. For more links to help you get started, check out our documentation and helpful links page.

 

Driver Support

We recommend upgrading to the latest drivers for the best performance.

AMD: AMD has released optimized graphics drivers supporting AMD RDNA™ 3 devices including AMD Radeon™ RX 7900 Series graphics cards. Download AMD Software: Adrenalin Edition 23.5.2

Intel: Developers interested in Intel drivers supporting Stable Diffusion on DirectML should contact Intel Developer Relations for additional details

NVIDIA: Users of NVIDIA GeForce RTX 20, 30 and 40 Series GPUs, can see these improvements first hand, in GeForce Game Ready Driver 532.03

 

Get hold of us

The AI space is changing fast! In case you run into any problems feel free to open an issue on our Github repo or email us at askdirectml@microsoft.com

 

 

0 comments

Discussion is closed.

Feedback usabilla icon