What is Microsoft Mesh?

Microsoft has introduced its newest mixed-reality platform, Microsoft Mesh, at the Ignite digital event in March 2021. Computing technologies came a long way through mainframes, personal computers, and smartphones. Today, mixed-reality evolves as the mainstream technology by changing from screen-oriented devices to collaborative cross-platform, virtual, and augmented experiences. 

Microsoft Mesh connects the physical and digital worlds, allowing users to transcend the traditional boundaries of space and time. In Microsoft Mesh enabled applications, users can feel the real presence, and they can connect from any device of their choices, such as AR, VR, PC, or phones. For developers, Microsoft Mesh can be used to build immersive multi-user cross-platform mixed reality applications that leveraging AI-powered tools for spatial maps, immersive presence holographic rendering, and multi-user sync. Users can also leverage Microsoft Mesh to conduct visual design sessions, enhance virtual meetings, host virtual social gatherings, and much more with a geographically diverse set of participants. In addition, it allows users to interact more easily with others and collaborate on both 2D and 3D content.

Microsoft Mesh evolves as a new collaborative platform, and it builds natively on top of Azure.  It enables developers to create Mesh enabled applications that promise to experience three things.

• Allows users to experience the mixed reality together. For example, imagine users share holograms, manipulate them and be in the same space where the holograms share between all of them.
• Provides agency to users in a mixed reality that allows them to represents themselves from avatars to holoportation. With that, users can experience the mixed reality together in a social way even they are not in the exact physical location. 

• Allows users to connect from anywhere using any device such as augmented reality devices like HoloLens, virtual reality devices like Oculus or Windows virtual reality headsets, and even 2D devices like PCs or mobile phones.

Experiencing the Microsoft Mesh

Microsoft has released the preview experience of Microsoft Mesh for HoloLens 2. In the future, several applications will take advantage of Microsoft Mesh both from Microsoft and other ecosystems of partners. In addition, it enables users to join the experience from anywhere.

Image source:  Microsoft Ignite - YouTube

Foundation to Microsoft Mesh is the notion of spaces. Think of a space like a conference room except for something that persists across time and geographies. For example, let’s imagine you enter a space called pipeline review space where your colleagues gather to discuss the ongoing pipeline project. Inside the conference room, each participant represents avatars, and voices can hear as if you talk to someone across the room. As you lift your hand, you can see a menu to add more people into space. You can change your avatar settings, and you can annotate, draw, bring in more shapes and figures into space, etc. You also can bring in 2D and 3D content to collaborate upon. Microsoft Mesh integrates with Azure Active Directory (AAD) and Microsoft Account (MSA), and that allows hosts to invite participants into designated spaces in a simple but trustworthy manner.

Image source:  Microsoft Ignite - YouTube

One of the significant aspects of Mesh is the integration with Microsoft 365 Graph that enables to transfer of a wide range of data inside Mesh’s 3D space that includes user connections, collaborative efforts, workflows, and documents, etc. For example, to review the pipeline project, you can bring a 3D model stored in your One Drive to the Mesh space. In addition, one of the essential things about collaboration in mixed reality is the three-dimensional fidelity that it offers. While each participant can have a unique perspective into the model, anyone can rotate, translate, zoom, add or redact from this model, driving clarity to the decision making while reducing the cost and complexity of presence in-person meetings.

Image source:  Microsoft Ignite - YouTube

In in-person meetings, participants can use whiteboards or papers to annotate what they are explaining. With Microsoft Mesh enabled applications, users can turn their hand to a pen and annotate in the space. One of the benefits of collaboration in Microsoft Mesh is persisting these spaces across time and geographies. If the model requires some changes, the engineering team can join the space and suggest their ideas to improve the model. 

More underlying capabilities

For Microsoft, it has taken years of work to bring the people virtually together. With Microsoft Mesh, privacy and security concerns are the top priorities ranging from integration of identity services, encryption across all data and in transit. At the core of the platform, Microsoft Mesh hooked into identity with options for Azure Active Directory and Microsoft accounts. It also uses Microsoft Graph to make it easy not to connect the users but bring in content and activities from their 2D to 3D medium. All of this runs in Azure, and users don’t need to worry about provisioning backend services and live stage sync services. These provide building blocks to light up mixed reality collaborative applications with four main AI pillars, namely spatial maps, 3D presence, volumetric rendering, and multi-user sync. Spatial maps use to anchor holographic content across time, devices, and users. 3D presence uses to bring people together into the virtual or physical space, and in this case, Mesh covers both ends of the present spectrum from representative avatars to photorealistic holoportation. Microsoft Mesh provides these features with a wide range of choices across avatar rigs, capture devices, and consumption endpoints. The volumetric rendering or holographic rendering for uncompromised fidelity and details transcend the intelligent edge and the intelligent cloud. Finally, multi-user sync enables the physical and spoken actions, and holographic transforms that users share in real-time.

Microsoft Mesh is a multi-device support platform. It supports a range of devices depending on the user’s requirements. For example, if users prefer a 3D volumetric experience, they can use a Microsoft HoloLens, HP Reverb G2, or Oculus Quest 2. For 2-dimensional views, users can use smartphones, tablets, or PCs. As mentioned earlier, Mesh allows users to connect from anywhere to any of their preferred devices.    

Future of Microsoft Mesh

Mixed reality is a new medium of computing, and it is the future of computing. Therefore, it can expect that Microsoft Mesh will open up many potential possibilities and transform the future of the workplace. Many enterprises have already decided to take advantage of this newest technology. As an example, 50% of Fortune 500 companies by now purchased HoloLens. So it’s sensible to think that the mixed reality is transformative in enterprises today. But as a medium, it will be transformative to everything and everyone in the future.

Microsoft Mesh experience for HoloLens is in preview at microsoft.com/mesh, and for developers, more guidance can find at aka.ms/MeshDocs.