WSLg Architecture

Steve Pronovost

Steve

If you landed on this blog, you’ve probably seen our announcement for GUI applications support in the Windows Subsystem for Linux being available to Windows Insiders and looking for more details on how WSLg was built. If so, you’ve come to the right place!

Be warned that this blog is fairly long and technical. We wanted to tell WSLg’s story, not just the architecture we picked, but also the reasons we made the various choices we made. We hope you find this behind the scenes informative and interesting.

Philosophy and early goals

When we started looking at supporting GUI applications in WSL, we quickly decided that we wanted to support both X11 and Wayland applications. Almost all applications that our users were asking to run within WSL were X11 based, but as the Linux desktop community was moving toward Wayland, we felt it was important to support it. We didn’t want Linux on Windows to be stuck in the past, limited to X11 applications, and for WSLg to be a hindrance to the shift to Wayland.

It was also important for us to build a Linux applications desktop environment that closely followed standards. We wanted applications to run as-is, without the need for any modifications. We didn’t want Linux applications to have to adapt or change their behavior to run within WSLg, we wanted WSLg to follow all Linux desktop standard thoroughly, so things just work. We saw this as an all-around win-win. It avoids fragmentation and means better applications compatibility, which leads to happier users. We probably messed up a thing or two 😊… if you come across those, please don’t work around them, let us know by opening an issue on the WSLg GitHub project so we can fix them.

We also wanted WSLg to be open-source and ensure that all communications between Linux, running in the WSL 2 virtual machine, and the Windows host followed either documented standard or defined in open-source code available on both end of the communication channel. No secret sauce allowed. We wanted to enable developers in the community to tinker with WSLg if they wanted to. Our WSLg project on GitHub has an architecture overview and details on how to get started building and running private versions of WSLg.

In terms of the user experience, we wanted to offer a unified and integrated desktop experience. An experience which allowed Linux and Windows applications to coexist, side-by-side, on a single unified desktop and where applications behaved in a predictable way. There are plenty of solutions out there that offer desktop-on-desktop style experience. We wanted WSLg to feel seamless, to fade in the background, and to let developers focus on their job, using whichever Linux or Windows applications works best for them, without hassle. The preview offers a pretty good experience, but still has various limitations that can cause some distractions. For example, the preview still uses server-side window movement and resizing, resulting in window move and resize operations which don’t feel as smooth as native, which also results in the inability to snap Linux windows on the edges of the monitors or to custom snap region. These annoy us too 😊. We will continue to improve the experience over time and reduce the gap in behavior or performance between how Linux and Windows applications behave, while ensuring we continue to design these solutions following our core principles.

Choosing a path: Building on Weston

We decided to build WSLg as a Wayland first Linux desktop and to support X11 applications by hosting the XWayland server that the xorg community built for that purpose.

The big question for us was: where to start from to build this Wayland compositor? Do we write a new compositor from scratch? Do we remote the Wayland protocol over to the host and run brand new compositor on Windows itself? Or should we build on top and contribute to an existing project?

This was a brand-new space for us and felt we needed perspective from wiser folks in the Linux community. We weren’t quite ready to announce to the whole world that we wanted to enable Linux GUI applications on Windows, so we reached out to a few folks we had worked with and trusted. We would particularly like to thank Kenneth Clark, an especially helpful and active member of the WSL GitHub community. Kenneth helped us build the very first proof-of-concept protype, that was shown at //build2020, for what would later become WSLg. And Daniel Stone, from Collabora, who we have worked with extensively on the D3D12 gallium driver for Mesa project. Daniel’s insight, perspective and extensive knowledge of Wayland and Weston were critical in helping us properly understand our choices and ensuring WSLg started on a solid foundation.

So why did we decide to build on Weston?

In a nutshell we felt it was the best approach that allowed us to build on top of what the community had already built and ensuring that we have a compositor which was as compliant as possible… what better way of achieving this than running the official Wayland reference compositor!

Weston is the heart of WSLg. The front end of the Weston compositor, which defines and implements the various Wayland protocols, is effectively unmodified outside of bug fixes or to accommodate new paradigms related to application remoting. WSLg doesn’t add any new or private Wayland protocols out of Weston, as far as Wayland applications (or XWayland for X11 applications) are concerned, they are interacting with Weston. One way this can be observed is through app compat… we’re pretty close to parity with native Weston running against the drm backend. Typically, when an application works correctly in Weston native, it also works correctly in WSLg and vice versa.

We feel that’s a great position to be in. As we fix app compat issue for WSLg and upstream our fixes, we help make Weston (and Wayland) better. As the community pushes fixes into Weston, we also directly benefit from these fixes. Getting to great application compatibility is going to be a long journey, but one we feel well aligned with the Wayland community by building on top of the Wayland project reference compositor.

Weston already had an RDP backend that allowed it to communicate with a host through the Microsoft standard Remote Desktop Protocol (RDP) using FreeRDP. Extending the existing Weston RDP backend to teach it new tricks sounded quite interesting to us. On the Windows side of things, we have a lot of experience leveraging RDP to remote applications. We have Windows Virtual Desktop (WVD), a world scale service running in Azure and streaming Windows applications to users worldwide. WVD uses RDP RAIL (Remote Application Integrated Locally) technology to integrate these remote applications in the user local desktop experience. And we have Windows client technologies, such as Windows Defender Application Guard for both Edge and now Office, which leverages a variant of this RDP technology, called VAIL (Virtualized Application Integrated Locally), optimized for transport over VM boundaries instead of over a network.

Extending Weston to teach it about application remoting and extending the RDP backend to leverage both RAIL and VAIL meant we would have a generic solution built from the ground up with network transparency in mind, on a protocol that is already widely used in the industry and operates at scale, this sounded quite appealing.

WSLg Architecture

The heart and soul of WSLg is the Weston compositor. This is the standard Weston compositor with a heavily expanded RDP backend, a new RAIL/VAIL shell and various bug fixes here and there. At the moment, we’re building these components from a project mirror while we work on upstreaming our contribution back to the respective projects. Our goal is to eventually build WSLg from purely upstream components, making WSLg a great and simple production environment for folks wanting to tinker with Wayland or Weston.

We expanded the RDP backend in many ways. We’ve added support for applications remoting using the RAIL and VAIL (aka GrfxRedirection) protocol. It is now possible for the RDP backend to remote individual windows instead of the whole desktop. The difference between RAIL and VAIL is that RAIL copies a window’s pixel content over the RDP transport and is optimized for an RDP server (WSLg) and an RDP client (on the host) connected over a network. VAIL is very similar to RAIL, but is optimized for transport over a VM boundary. VAIL uses shared memory between the host and guest VM to avoid expensive pixel copies over the RDP transport. As part of building WSLg we’re making the VAIL/GrfxRedirection protocol public.

In addition to application remoting, we’ve also extended the RDP backend to support multi-monitor configurations, including support for per-monitor DPI scaling. The DPI scaling uses a combination of native Wayland support for the scale factor supported natively by Wayland, and RDP client-side scaling for scale factor which are not supported. This ensures Linux applications are always properly scaled based on the preference the user selected in the Windows UI settings. We added support for the clipboard so it’s possible to cut/paste text, html and bitmap data between Linux and Windows applications. Drag and drop is not currently supported.

We added support for both audio in and out. For audio we chose to run a PulseAudio server. We wrote small sink and source plugins for pulse which shuffle the audio data between PulseAudio and the RDP backend such that the audio streams can be integrated over the RDP transport to a local or remote RDP client.

WSLGd is a small, daemon like, application which is the first process to launch in the WSLg environment and who launches Weston, Pulse, establishes the RDP connection to the host then monitors these and restarts them if they ever crash or stop working.

We wrote a small RDP plugin which deals with integration of WSLg in the Windows start menu. The part of the plugin inside of Weston enumerates applications installed on the user distro, looking for desktop files. It sends that list of applications, alongside command line to launch them and icon to represent them, to the host portion of the plugin which adds these applications to the user’s start menu, allowing Linux applications to be launched directly from the Windows start menu. A few standard desktop file locations are monitored. When a user installs or uninstalls an application in their Linux distro, the operation is reflected a few seconds later on the Windows host.

Finally, we expanded FreeRDP to support new protocols and fixed a few compatibility issues when communicating with mstsc. Weston was already using FreeRDP for its RDP support and we didn’t see the need to move to a different solution, opting to push a few enhancements to FreeRDP instead, which can be used in other context beyond WSLg as well.

System distro

You may have noticed in the picture above this new thing that we call the system distro. You can think of the system distro as a containerized Linux environment in which we are running Weston and friends and projecting the various server sockets back into the user distro. The user distro is configured by default to reference those servers for X11, Wayland and audio support.

We decided to go with this approach for WSLg as it allows us to isolate WSLg from the user distro. We can service it independently of the user distro and enables us to offer a consistent experience across different Linux distributions. As we introduce WSLg, we expect to update it frequently over the coming months as we continue to add new functionality, improve performance, polish the experience, and fix applications compatibility issues. For users wanting to tinker with the system distro, we provide mechanisms for users to run private versions (see our WSLg contributing page)

For the system distro, we decided to use CBL-Mariner to host WSLg. CBL-Mariner is a lightweight and customizable Linux distribution maintained by our Linux System Group and allows us to centralize the maintenance of our Linux environment across various parts of Microsoft and ensure we remain current and on top of any security vulnerabilities or other important patches we need to pick up. Until now, CBL-Mariner had been focused on headless, containerized type workload running in Azure or edge products and services. Working closely with the CBL-Mariner team, we published various UI related packages to an official CBL-Mariner RPM repo to enable WSLg.

WSLg project on GitHub

If you would like to see even more in-depth details about WSLg architecture or look at the source code and build it yourself, please visit our official WSLg project page on GitHub.

Hardware accelerated OpenGL

We previously announced support for virtual GPU in WSL, which enables popular compute APIs to be available in WSL at near native performance. This is in addition to Microsoft’s own DirectML backend for Tensorflow which enables AI training in WSL across a broad set of hardware.

In addition to vGPU, we’ve been working with the Mesa community on bringing up a new d3d12 gallium driver for Mesa and recently announced the availability of this work on Windows, bringing support for hardware accelerated OpenGL and OpenCL to ARM based Windows PC which previously lacked this support.

Linux support for the new d3d12 Mesa driver was upstreamed to Mesa a few weeks back and is part of the official Mesa 21.0 release. This is the culmination of a long journey and enables hardware accelerated OpenGL applications in WSLg.

It’s important to note that WSLg doesn’t have a dependency on the availability of vGPU or accelerated OpenGL. WSLg works great for simple 2d applications with or without OpenGL acceleration. But if you’re trying to run more complex 3D applications, such as Blender or Gazebo, you will get a much-improved experience running on a system with a GPU supporting WDDMv3.0, which comes standard with WSL vGPU support and will automatically light up this OpenGL acceleration through Mesa.

You can find preview WDDMv3.0 drivers from each of our partners using the links below. These drivers will eventually be shipping on Windows Update and along with new system when the next version of Windows is released.

In terms of Mesa, most distribution today are still on older 20.x versions. We’ve reached out to various WSL Linux distribution publishers to ensure an update to Mesa 21.x is on the horizon and that they update their Mesa package definition to build and include the new d3d12 gallium driver.

Depending on when you read this, accelerated OpenGL in WSLg may or may not immediately light up on your system as you may still be running an older version of Mesa. If you don’t want to wait on your distro to pick up those changes and want to try this out immediately, you can visit Mesa official home and build a private version of Mesa with this support. Alternatively, you can try out Canonical recently announced Ubuntu on Windows Community Preview for WSL 2. This new distribution of Ubuntu is built from bleeding edge components an includes support for Mesa 21.0, enabling seamless hardware accelerated OpenGL with WSLg out of the box. This distribution is meant for advanced WSL users to help test and debug upcoming features and not meant as a “daily driver” but is a great way of getting an early glimpse at those features.

A note about performance as we’re sure some folks will want to compare native versus WSLg version of applications 😊. For WSLg v1, Mesa interops with Weston through system memory. On discrete GPUs this means that rendered content needs to be copied to system memory before being presented to the compositor, to be brought back onto the GPU in the RDP client running on Windows. This cost scales with the frame rate, and application running at super high frame rate will see a more significant impact than applications running at more reasonable frame rates. For integrated GPUs, the data doesn’t have to leave system memory as it is shuffled across, however presents through Mesa are currently synchronous, meaning there is a small bubble on every frame (wait for render, push the rendered frame, start the next one) which has a performance impact.

Here’s a totally unofficial snippet of performance taken from two of my PCs to put a perspective on things, first on a discrete GPU running a very high frame rate (a particularly bad case), second on an integrated GPU (a particularly good case). This is for Geeks3D GpuTest running piano.

Native Win32 (Mesa)

WSLg (vGPU – Mesa)

WSLg (Software – Mesa)

NVIDIA RTX 3090

(Yeah, super lucky to have friends in right places😊)

540fps

350fps

4fps

Surface Book Gen3 (Intel – GPU)

19fps

18fps

1fps

Although there is a hit in performance due to having to do system memory interop, the result is still much better performance than software rendering. Closing the performance gap between native win32 application and Linux applications running in WSLg is something we want to improve for WSLg v2 and beyond, but for v1 we wanted to focus our energy on the core experience while still offering good performance, even if not native.

Feedback

We’re looking forward hearing your feedback on WSLg, what works well and maybe not so well. If you encounter problems or want to offer suggestions, please open an issue on the official WSLg project page.

21 comments

Leave a comment

  • Avatar
    Hristo Hristov

    Intel vs NVIDIA? How come Intel provides much, much, much more stable FPS? 🙂

    What’s in for WSLg V2 – modern Vulkan and PipeWire support?

    • Steve Pronovost
      Steve PronovostMicrosoft employee

      It’s not really an Intel vs NVIDIA thing as much as integrated versus discrete GPU thing.

      For WSLg v1, Mesa interop with our instance of Weston/XWayland through system memory. On a discrete GPU this mean we’re rendering on the GPU… then copying from the rendered frame in the dGPU VRAM back to system memory (over PCIe), than presenting to the compositor… and once on the Windows side of things, the data gets back into VRAM… this adds overhead and this overheads scale with the frame rate… so the faster you go, the more overhead you have as a %. So on a 3090 running at very high frame frate, the impact of this system memory interop is pretty severe. On an integrated GPU like Intel, the data remains in system memory the whole time, so there is less overhead per frame… also since the overhead scale with the frame rate and that we start at a pretty low frame rate… the end result is pretty close. This is an area we want to improve in the future with v2 and beyond. We had share some plan around this at XDC, but scale back from those for v1.

      Vulkan and PipeWire are definitely on our list of things we would like to enable as well at some point in the future :-).

  • Avatar
    Ed Borasky

    So far the only issues I’ve run into are with the Ubuntu on Windows Community Preview for WSL 2. It looks like it’s based on “Hirsute”, which neither NVIDIA nor the Comprehensive R Archive Network (CRAN) are currently supporting. So I’ll have to forego Mesa 21 until they catch up.

    • Avatar
      Sanjeev Gupta

      Ed, I am running the Ubuntu Preview. This is what I see:

      root@T450s-W10:/home/sanjeev# apt show mesa-utils
      Package: mesa-utils
      Version: 8.4.0-1build1
      Priority: extra
      Section: universe/graphics
      Source: mesa-demos
      Origin: Ubuntu
      Maintainer: Ubuntu Developers 
      Original-Maintainer: Debian X Strike Force 
      Bugs: https://bugs.launchpad.net/ubuntu/+filebug
      Installed-Size: 150 kB
      Depends: libc6 (>= 2.14), libgl1, libx11-6
      Homepage: https://mesa3d.org/
      Task: xubuntu-desktop, ubuntu-mate-core, ubuntu-mate-desktop, ubuntu-budgie-desktop, ubuntu-budgie-desktop-raspi
      Download-Size: 34.2 kB
      APT-Sources: http://archive.ubuntu.com/ubuntu hirsute/universe amd64 Packages
      Description: Miscellaneous Mesa GL utilities
       This package provides several basic GL utilities built by Mesa, including
       glxinfo and glxgears.

      My /etc/apt/sources.list points to Hirsute .

    • Steve Pronovost
      Steve PronovostMicrosoft employee

      The NVIDIA driver used in WSL 2 isn’t the same one that is used on a native Linux installation. vGPU support in WSL2 is provided by your Windows GPU driver.

      If you install the following driver from NVIDIA (https://developer.nvidia.com/cuda/wsl/download), it should work well and right out of the box with the Ubuntu Windows Community Preview for WSL 2 / Hirsute. Alternatively, you can compile and run a private version of mesa from https://github.com/mesa3d/mesa until the more mainstream version of Ubuntu pick up Mesa21.x.

      • Jiwen Cai
        Jiwen Cai

        Hi I just compiled mesa3d. It seems I got 21.x version, but still don’t have vGPU support (i.e. GPU test is still running really slow). Wondering whether there is any configuration i should use when compiling mesa?

        $ glxinfo | grep version
        server glx version string: 1.4
        client glx version string: 1.4
        GLX version: 1.4
        Version: 21.0.3
        Max core profile version: 4.5
        Max compat profile version: 3.1
        Max GLES1 profile version: 1.1
        Max GLES[23] profile version: 3.2
        OpenGL core profile version string: 4.5 (Core Profile) Mesa 21.0.3
        OpenGL core profile shading language version string: 4.50
        OpenGL version string: 3.1 Mesa 21.0.3
        OpenGL shading language version string: 1.40
        OpenGL ES profile version string: OpenGL ES 3.2 Mesa 21.0.3
        OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
        GL_EXT_shader_implicit_conversions, GL_EXT_shader_integer_mix,

  • B. C.
    B. C.

    Today, after update to Windows 10 Insider Preview Build 21364, WSL doesn’t work any more.
    I see in terminal “The parameter is incorrect.”
    How to fix?

    • Avatar
      Алексей Евдокимов

      Seemingly they messed up something with memory layout, so it doesn’t currently work on older Intel CPUs with 36-bit physical address space, i.e. Ivy Bridge. I’m not even sure if it works on Haswell, didn’t have time to try.

    • Steve Pronovost
      Steve PronovostMicrosoft employee

      Yeah, that’s a known issue on older processor, a fix is on its way.

      At the moment you’ll need to either uninstalled WSLg (Add/Remove Program -> Windows Subsystem for Linux WSLg Preview) or disable gui applications in our .wslconfig file (e.g. c:\users\spronovo.wslconfig).

      [wsl2]
      guiApplications=false
      
  • Shixin Zeng
    Shixin Zeng

    Is vGPU only supported on select intel GPUs? I have a i3-6100u with HD 520 and driver 28.20.100.8322, in Ubuntu on Windows Community Preview for WSL 2, I’m still not able to see hardware acceleration:

    ~$ glxinfo -B
    name of display: :0
    display: :0  screen: 0
    direct rendering: Yes
    Extended renderer info (GLX_MESA_query_renderer):
        Vendor: Mesa/X.org (0xffffffff)
        Device: llvmpipe (LLVM 11.0.1, 256 bits) (0xffffffff)
        Version: 21.0.1
        Accelerated: no
        Video memory: 5869MB
        Unified memory: no
        Preferred profile: core (0x1)
        Max core profile version: 4.5
        Max compat profile version: 3.1
        Max GLES1 profile version: 1.1
        Max GLES[23] profile version: 3.2
    OpenGL vendor string: Mesa/X.org
    OpenGL renderer string: llvmpipe (LLVM 11.0.1, 256 bits)
    OpenGL core profile version string: 4.5 (Core Profile) Mesa 21.0.1
    OpenGL core profile shading language version string: 4.50
    OpenGL core profile context flags: (none)
    OpenGL core profile profile mask: core profile
    
    OpenGL version string: 3.1 Mesa 21.0.1
    OpenGL shading language version string: 1.40
    OpenGL context flags: (none)
    
    OpenGL ES profile version string: OpenGL ES 3.2 Mesa 21.0.1
    OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

    I can see d3d12 driver exists in the system:

    ~$ ls /usr/lib/x86_64-linux-gnu/dri/
    d3d12_dri.so  i965_dri.so  kms_swrast_dri.so  nouveau_vieux_dri.so  r300_dri.so  radeon_dri.so    swrast_dri.so      vmwgfx_dri.so
    i915_dri.so   iris_dri.so  nouveau_dri.so     r200_dri.so           r600_dri.so  radeonsi_dri.so  virtio_gpu_dri.so  zink_dri.so

    However, I don’t see the dxg device (which I believe is the vGPU file?):

    ~$ ls /dev/dxg
    ls: cannot access '/dev/dxg': No such file or directory
    • Steve Pronovost
      Steve PronovostMicrosoft employee

      At the moment the only Intel supported driver for vGPU in WSL is the one linked in the blog (https://downloadcenter.intel.com/download/29526). No other Intel driver exposes that functionality. Since /dev/dxg is missing, it indicate your driver does not support vGPU. Trying installing the one from the blog. We’re working with Intel to get new driver posted and eventually this will all be on Windows Update (for supported devices), so stay tuned for more driver options.

      • Shixin Zeng
        Shixin Zeng

        Thanks for the reply. Yes, That’s the driver and the exact version I installed (28.20.100.8322), which made me wonder if it’s because my iGPU is not supported, or am I missing anything?

    • Avatar
      Eric Pellegrini

      Experimental WSL+Nvidia CUDA support actually came well before graphics support. Applications using CUDA should not have any problem.

      • Avatar
        Arnon Marcus

        I’m asking about the intersection of both : CUDA-usage from within a Linux desktop UI application.
        Say I have a linux desktop app that manually handles the drawing of it’s content (it’s frame-buffer) using CUDA/OptiX, then displaying that in the app’s window using whatever API that’s native to the linux distribution.
        The drawing of the window’s “chrome” (borders, title-bar, etc.) I suppose would be handles by WSLg,
        but the internal content of it, the app would want to draw by itself, so – what if it wanted to employ CUDA in doing so?
        What’s going to be the interop/architecture there?
        How many times would the pixels be expected to be copied-over in-memeory and/or across the PCIe-bus?
        Will it be able to run CUDA kernels on the GPU at (/near-)native-speeds?
        How seamless would the development experience be in such a scenario?
        etc.

        Hope this clarifies what I’m asking.

  • Avatar
    Alwin Hendriks

    Hi Steve,

    Great article. I have a question related to all this GPU and RDP stuff and I hope you might be the person to ask.
    In our environment we are doing some test with WSL2 and CUDA because this is way faster that running this on Windows only and removed the hassle from having to have a full Linux host. Anyway the issue we ran into is graphics support via RDP connections.
    Our setup is a workstation with a powerful graphics card and running WSL2 with the CUDA drivers, locally everything works but if we use RDP to connect to this workstation WSL2 and the CUDA drivers can’t locate the GPU anymore. The GPU is a quadro P620 so I am assuming it has more RDP support than the regular gaming cards have but it still fails. Do you perhaps have any insight on this? I find it hard to tell if this touches on your work as you are speaking of using the RDP protocol to get things done but on the other hard it seems to be a more local use of the RDP protocol.

    In any case thank you for your time.

    Cheers,
    Alwin