Building immersive experiences with Microsoft Mesh

Tyler Adams

October 5th, 20231 5

Microsoft Mesh, now in public preview, is a new 3D immersive experience for the workplace, helping virtual meetings and events feel more like face-to-face connections. Now you can bring your 3D world building skills to the global workforce, helping users connect like never before. With Mesh toolkit you can create fully customized, multi-user 3D experiences – giving you a powerful, imaginative medium for addressing business needs and delighting users.

At a time when companies are re-imagining the way employees come together, Mesh adds a new dimension to the solution space, with innovations that help organizations manage the opportunities and challenges of the distributed workplace. Immersive experiences are uniquely suited to facilitate deeper human connections across physical distance, visualizing models and simulations, envisioning spaces that are difficult to access, and immersing people in stories that capture the imagination. With Mesh, organizations can elevate team building, training, product demos, collaboration, social connections, and more.

Mesh toolkit makes tapping into the value of immersive technologies easier than ever by reducing time to launch and lowering the barriers to achieve security and scale delivering experiences customized to your business requirements. In Microsoft Mesh, the immersive content you create is experienced by users on a desktop PC or in VR. Mesh does the heavy lifting to enable engagement in 3D environments by handling the avatar system, Spatial Audio, and multi-user synchronization. Development, iteration, performance optimization and publishing are facilitated through the Mesh toolkit. The toolkit supports Unity so you can build environments in a familiar way and leverage existing scenes and assets to build experiences in Mesh. You can also enrich your experiences by adding interactive content, inserting logic, integrating with business data, or infusing AI – all with physics and interaction tools that you can leverage out-of-the-box.

Start your Mesh development journey now and read on to learn how it will be easier, faster, safer, and even more fun to bring your visions to life.

What can I expect from Mesh?

Multi-user experiences stay in sync and give users a sense of being together

Mesh handles many of the biggest challenges to giving people a sense of co-presence in immersive spaces. Mesh is multi-user by default, receiving input from all users and applying changes to all clients so users stay in sync and can share an experience together. Spatial Audio enables a rich audio landscape that matches the directionality of sound in the physical world so participants can sense where others are in the 3D space and hear the virtual world around them. This helps participants feel fully immersed and enables multiple discussions and side conversations to happen in the same place at the same time.

Mesh’s avatar system is highly customizable so users can represent themselves the way they want when connecting with others. Avatars are tied to a user’s existing work identity through Azure Active Directory (AAD) so there’s no need to create separate credentials. In fact, the same avatar system is used in Microsoft Teams so hundreds of millions of global users have an easy onramp to set up and get familiar with using their avatars.

Enterprise-grade security and privacy built on Microsoft 365

Mesh was built from the ground up for use in enterprises scenarios so it can support immersive solutions in the places where people work. Security, privacy, and access are handled similarly to other business applications in the Microsoft 365 admin center giving you flexible control of company data. There is no need for installations outside of a company’s typical requirements, and users never need to leave the corporate network.

Immersive experiences run on PCs, VR headsets, or both

Currently, Mesh can be experienced on desktop PCs and Meta Quest VR headsets. Depending on your business need, you may want a higher-resolution scene that is optimized for powerful PCs. Or you could build scenes optimized for Quest VR devices and choose to add PC as well. In the latter case, all users can interact seamlessly in an event no matter which hardware platform they use. This flexibility can also eliminate or reduce the need to equip users with new hardware, lessening the overall investment for exploring immersive solutions.

Content and animations you build integrate with the event organizer’s workflow

Immersive environments you upload to Mesh can be made available for use in events like all-hands meetings, showcases, training sessions or team building activities. You enhance event organizers’ ability to deliver powerful interactive experiences by exposing content and animations they can activate during a live event. Event organizers can play, pause, loop and stop content during events so that each production is unique and expertly timed.

Rich tools amplify your creativity and meet the needs of the business

Take a look at some of the capabilities offered in the Mesh toolkit that make it easier to build, iterate, optimize and publish rich immersive experiences:

Add natural interactivity with Mesh visual scripting

Add interactivity, connect triggers to actions and insert custom logic to your scene with Mesh visual scripting. Visual scripting runs client-side to support simple interactivity without writing code letting you build custom low-latency effects. It also works without a server deployment, making it simple to get started.

Mesh visual scripting is based on Unity visual scripting, and includes features designed for Mesh development. With it, you can create industry-standard Unity visual script graphs, which run in the Mesh Visual Scripting execution environment. The execution environment works behind the scenes, tracking and synchronizing local property changes across clients to ensure a consistent shared experience. Sandboxing protects users from illicit attempts to access sensitive local data or read or write arbitrary remote data on other clients.

Access your data and the power of .NET with Mesh cloud scripting.

Introduce live backend data from your company or public sources, infuse AI, or add complex logic with Mesh cloud scripting. The scripts you write in C# are .NET apps and can call any .NET API, access NuGet libraries, and leverage best-in-class tools to power your experiences.

Cloud scripts are written against the Mesh API and run on your cloud server so updates happen quickly and accurately across all participants. The server delivers an authoritative version of the scene so changes are mirrored consistently to all clients and mapped directly to the Unity scene. This method allows for code reviews, diffs, history inspection, Git integration, access to external assemblies, and easy debugging at runtime.

Both visual scripting and cloud scripting work on their own or in combination with one another in Mesh.

Enable engaging, dynamic content with Mesh Physics

Mesh Physics provides a productive workflow for creators and a high quality experience for users. Mesh Physics is based on local authoritative simulation, which results in a very simple architecture. There’s no need for server-side logic or special logic on the clients. Most importantly, this approach results in a very responsive and low-latency experience for each player interacting with their environment, even when connected to a high-latency network.

With the Mesh Physics system, you can make your creations act in the realistic – or fantastic – manner that is consistent with your concept. Use Mesh Physics to control how gravity and other forces affect objects by making them bouncy or sticky. Allow users to grab and hold objects, teleport them, or make them explode. Add physics to your interactive experiences and games and inject limitless interactivity, creativity, and fun for participants.

Bring rich web content into your 3D environments with Web Slates.

Add one or more Web Slates to a scene to view dynamic dashboards, web pages, photos and videos. Drag and drop the WebSlate prefab, and then use scripting to connect your maps, diagrams, and data to the surrounding environment. With built-in input support, users can interact with web content intuitively across devices without extra development time.

Ensure high performance with optimized environments designed for Mesh

Create assets for use in Mesh using familiar 3D modeling software such as Maya, 3DS Max, Cinema 4D, ZBrush or Blender. Our detailed design guidelines will set you up for success with examples and guidance built from experience on topics such as project structure, modeling, texturing, lighting, and post processing. Graphics tools included in the Mesh toolkit provide code, shaders, assets, and samples to help improve the visual fidelity of environments while staying within performance budgets.

The content performance analyzer catches performance bottlenecks during authoring by identifying content optimization opportunities and automatically correcting them. The visual profiler validates performance at runtime by monitoring draw calls and CPU/GPU usage. Leveraging these tools to optimize performance will accelerate downloads, increase user comfort, and even improve battery life and fidelity—all without adding significantly to your dev time or affecting security.

Accelerate iteration with multi-user simulation

Get an approximate preview of what your content will look and feel like when it runs in the Mesh app using Unity Play Mode. This greatly speeds up the iteration cycle during content development, allowing you to debug the logic of interactive content using built-in Unity tools. Play Mode can run multiple clients within the same process; thus allowing a single developer to easily get a first impression of multi-user scenarios in a convenient split screen view.

Publish to Mesh hassle-free directly from Unity

Once your scene is ready in the Unity editor, you are just a few clicks away from publishing to Mesh. You’ll use Mesh toolkit to save it as an environment in your Microsoft 365 tenant. Event organizers who have been granted access can create an Event based on your environment and invite participants to come and share the experience.

Get started today

The best way to learn about creating with Mesh is to dive in. The Mesh toolkit is free to download and will provide all the packages you need to start building. Create a new project from scratch, port an existing project built with Unity, or kick off with one of the many practical samples included in the Mesh toolkit.

The Mesh 101 sample is a great place to start. It includes a pre-built Unity project and step-by-step tutorial that will show you how to add Mesh features and create an interactive experience. Developed for those with intermediate to advanced Unity skills, the tutorial will give you hands-on experience setting up your dev environment, adding interactivity with Mesh scripting, and triggering animations with Mesh physics. You’ll be able to see your progress and simulate the multi-user experience in Unity Play Mode.

Anyone can build with the Mesh toolkit but to upload interactive environments and explore them with others, all participants need a Teams Premium license. If you and your users do not yet have Teams Premium, you can get started with a 30-day trial.

Here are some of the resources we think you’ll find most useful:

Start your Mesh development journey.
Learn step-by-step with the Mesh 101 sample and tutorial.
Subscribe to the Microsoft 365 Developer channel to see new video content as it is added to the Mesh development playlist.
Get updates on Mesh development, learn about new content and webinars, or connect with our team by subscribing to our Discussion Space on Microsoft Community Hub.

Thank you for being part of the Mesh journey. We can’t wait to see what you create with Mesh!