By Nik Sachdeva

A new buzzword percolating in many technical communities is “Metaverse.” The concepts around an emerging Metaverse are some of the most exciting developments in the convergence of business and technology. In this article, I share some of my thoughts on building Metaverse systems and their applications from an industry perspective. We are all learning about how to build these systems effectively and I hope to encourage those in product management, engineering, and data science roles to learn and share their perspectives.

The material in this article represents my own individual opinions, and I am not attempting to convey the views of my employer, Microsoft, or any other affiliates.

A Metaverse for industries

While much of the hype surrounding the Metaverse has been driven to date by companies like Meta and Nvidia with a focus on the consumer (a.k.a. social) market, there is a significant business opportunity for leaders from multiple industries to leverage Metaverse concepts and drive innovation within and across their industry scenarios.

We can already see examples of such patterns today. In manufacturing, a leading robot manufacturer is using Microsoft Azure services that can enable field workers, designers, analysts, and other specialists to work together and optimize remote monitoring and maintenance, set up robots and new manufacturing lines, and enable worker training through simulations.

Other examples exist as well. In healthcare, some organizations are optimizing their warehouse supply chains through multi-robot and fleet interactions. Some automotive organizations are running simulations for autonomous vehicles over millions of miles and enabling personalized customizations for their end users in a virtual space. Some retailers are enabling digital worlds to provide consumers with virtual reality / augmented reality experiences for clothing, fashion shows, and other apparel-related experiences, among many others.

A common underlying factor among all these scenarios is the management of data. Multiple services combine to enrich, curate, model, and manage data, which can then be leveraged to build intuitive three-dimensional and web-based experiences that lay the foundation for Metaverse solutions.

Defining the industrial Metaverse

The earliest Metaverse reference comes from Neal Stephenson’s 1992 book Snow Crash. In the book, Stephenson defines the Metaverse as:

“A computer-generated universe that his computer is drawing onto his goggles and pumping into his earphones. In the lingo, this imaginary place is known as a Metaverse.”— Snow Crash, by Neal Stephenson, page 22

From a software perspective, the term Metaverse is not a singular concept, but instead represents an evolution of multiple existing and innovative technologies that need to interconnect in a seamless manner to enable ubiquitous experiences for the end user. Here is my opiniated definition:

A Metaverse is a composition of loosely coupled distributed (and sometimes decentralized) subsystems that help accomplish business objectives through ubiquitous experiences and the convergence of physical and digital assets. It is not a product.

Several important aspects are at work here:

A Metaverse system model

A way to think about the Metaverse is to apply systems thinking and decompose the Metaverse into multiple subsystems. These subsystems manage assets and their interactions within the Metaverse. An asset is a data structure that can represent anything (a machine, a robot, a sensor, a human, an object, a system, and more). The subsystems enable data management in areas including communication, state management, identity, privacy, commerce, and object representation. The following subsystems can be considered as core components of a Metaverse solution. Note that these are a finite subset of all the potential of the Metaverse; I expect that these components will grow as this space matures.

Distributed fabric: This is the underlying distributed platform that will host the Metaverse subsystems and provide infrastructure capabilities (Platform Services, DevOps, DevSecOps, MLOps, and IaC). The fabric will also provide end-end security, proof of ownership, compliance, and governance services for Metaverse(s). Microsoft services such as Azure Compute, Azure Messaging, Azure Kubernetes and Edge, Azure Monitoring can enable this sub-system.

Integration connectors: Not everything will be available in the Metaverse, and existing systems will continue to work. We will still need in-house legacy, third-party, and open-source system data. The connectors will enable data push and pull services from existing systems while enabling an extensible data pipeline. Logic Apps, Azure Data Factory, and Microsoft Industrial IoT stack may provide these connectors.

Multiverse channels: There will be more than one Metaverse. We already see examples such as Decentraland, Minecraft, and MetaMetaverse. An Industrial Metaverse will have additional requirements around Intellectual Property (IP) compliance and security and will need mechanisms to share assets and monetize Metaverse assets (like the concept of a “port” from Snow Crash). Multiverse channels will provide the ability to securely interoperate and port these data assets across Metaverse(s). Azure Confidential Compute enables a foundation to share this data securely.

Commerce connectors: A Metaverse will need a mechanism to monetize assets and the commerce connectors will provide hooks to connect with payment gateways and digital wire protocols. Assets may be monetized or leased to other Metaverse(s). Another promising direction for this layer involves upcoming technologies such as Web3 and NFT, and we will have to see how they can become effective in enterprise industry scenarios.

Asset representation: Physical assets need to be virtually represented and their states captured in an event log for traceability. Technologies such as Azure Digital Twins can play a role in representing asset state data and making it available for other subsystems. Asset Behavior Services will enable the digital twin to interact with the physical world in a bidirectional way to enable telemetry, command and control, and remote monitoring functions. These services will also enable asset-asset interactions. Asset identity will play multiple roles. First, it will provide a secure identity for the asset within the Metaverse. Second, it will provide proof of asset ownership, which will be important to ensure that assets can be monetized or ported to other Metaverses but still not lose their original ownership. Finally, it will enable right permissions on the asset within a single world or across shared worlds. Azure Arc, IoT, and edge services as well as industry standards such as ROS (Robot Operating System) can enable these.

Asset world representation: To enable high fidelity of the physical environment and testing of an asset in multiple or new environments, asset world services will provide for the reproduction of world(s) within the Metaverse. Machine Learning may be used to generate these worlds and create new worlds with the domain context.

Rendering engine: To represent each asset in the virtual world, an underlying rendering engine will provide capabilities to create, upload, render, and update assets and worlds. Artificial Intelligence (AI) will play a key role in determining asset representation, mapping to various worlds. It will be important to support multiple interfaces including 2D web, mobile, and virtual reality and to support devices from current browsers to VR/AR/MR/XR interfaces such as HoloLens and Meta Quest products. Microsoft Power Platform and Microsoft Mesh may play a role in building these interfaces. It will also be important to leverage open standards such as USD, glTf, and WebGL to enable interoperability.

Simulation and synthetics engine: This will provide services to generate digital representations of assets and create completely new asset instances that have no existence in the physical world (e.g., for simulation). Machine Learning will play a key role through re-enforcement learning, generative algorithms to create assets and environments. Microsoft services such as Project Bonsai, Microsoft Synthetics, and Microsoft AirSim may be leveraged to play roles in providing these types of capabilities. Additionally, open solutions such as Gazebo that are popular within the robotics community can be leveraged.

Industrial Metaverse applications

Let’s look at a hypothetical example of how these subsystems can be leveraged to enable a smart factory scenario in the manufacturing industry. In manufacturing, both Operational Technology (OT) and Informational Technology (IT) need to converge to ensure productivity of many production lines and smooth supply chain operations. The amount of data generated from these systems is massive and field workers and operators require real-time insights and time-sensitive calculations for metrics such as OEE (Overall Equipment Effectiveness) to measure manufacturing productivity.

The distributed fabric and resilient channels subsystems provide services to enable scale and 24x7 factory operations at the equipment, factory, and downstream systems levels (which are usually cloud based). The integration connectors enable data from PLC, ERP, SCADA, CRM, and MES (see glossary) systems as well as interoperability with frameworks such as ROS and protocols like OPC-UA to push or pull data from the factory to the cloud and vice versa. Multiverse channels and commerce connectors enable organizations to securely share data and assets; for example, a robot manufacturer can lease a “robot asset” to a customer that includes its properties and data, which can then be used by another metaverse instance to perform operations or simulations on a physical robot.

Asset world representation enables a replica or digital twin of the factory to create a data model of how operations and performance can be monitored within a single or multiple factories. A world can be a small factory floor or a large multi-story electric vehicle plant, among other examples. Asset state, identity and behavior enable factory assets (e.g., boilers, forklifts, robotic arms, AGV, production show floor, and cells) to communicate in a bi-directional manner. For example, an automated system trigger might command a robot in the Metaverse to change its task configuration, run a simulation test, and cause the behavior service to communicate and implement the resulting physical state changes and then monitor the actions. These subsystems also enable constraints and privileges to ensure privacy and security. For example, if a robot is not allowed to enter certain premises of a factory in the real world, it should also not be allowed to do so during simulation in the virtual world.

Rendering engine services are responsible for the user experiences for different personas for the plant. For example, consider a plant manager who runs real-time data simulation on the digital twin of a factory to identify performance improvements. A field worker then leverages an AR (Augmented Reality) device to collaborate with a skilled worker to tune the performance remotely. A simulation and synthetics engine subsystem provides engineers and operators with the ability to simulate data at hyperscale and create synthetic environments and assets to train the algorithms and workloads.

Now here is the good news: Many of the subsystems in these examples exist today or are upcoming technologies, and many are already in use in building production solutions.

We don’t need a new Metaverse product — instead, the challenge will be to seamlessly integrate these existing and upcoming technologies to solve real-world business problems.

Challenges ahead: Where do we go from here?

As I’ve endeavored to describe, there are considerations that will be critical for implementing Metaverse concepts. Let’s discuss some of those:

Cost of ownership: Cloud providers and industry leaders are typically focused on reducing operating costs for products and services. Multiple Metaverses and three-dimensional rendering capability may lead to an increase in compute and memory usage, which may increase cost of ownership and management for first- and third-party industry products.

Sustainability: The number of interactions and the messaging between physical and digital worlds may lead to increases in the use of data centers and cloud infrastructure. The use of green data centers and green software engineering practices will play a role in reducing additional carbon emissions that Metaverses may create.

Data ownership: In the scenario I’m describing, a Metaverse is likely to become a hub for sharing and partnering within various industries. This will include asset creation, asset marketplaces, world designs, and application of skilled industrial knowledge. From a systems perspective, this is all just data. But who will own the data? It will be important to have granular data sharing to enable a co-operative metaverse.

Security, ethics, and compliance: As I’ve discussed, a Metaverse is a composition of subsystems, which means that security will be paramount for each communication boundary. Moreover, ethics and compliance will play a key role, and asset models will benefit from legal and ethics reviews to enable author authenticity and credibility. See the Metaverse Standards Forum plan to develop guidelines and specifications for this area.

Privacy: One of the most significant failures of a Metaverse could be if end-user and enterprise privacy is not ensured. Practices such as aligning user privacy standards and adhering to existing data protection and privacy policies (e.g., GDPR) will be important for human representation in a Metaverse.

Latency among worlds: An immersive experience requires near real-time conversations and so a latency lag within the current state of a machine can have a significant impact on machine productivity or cause threshold violations. While technologies such as 5G hubs and Wi-Fi 6 will help to minimize latency, we will also need solutions that can offload critical events to the device or to the edge and use other networks such as satellites. Azure for Operators provides services to help minimize this gap.

Ubiquitous experiences: While everyone is eager to implement three-dimensional models with VR/AR/MR, the world will not change immediately and so it will be important to support existing device interfaces such as kiosks, HMI, web, and mobile interfaces within an industrial setup. As I mentioned earlier, the human experiences we build upon need to align with multi-interface models.

Developer ecosystem: All the technologies that I’ve described cannot be fruitful if we do not have skilled engineering to leverage the right options to build the Metaverse. It will be important for organizations to invest in highly skilled engineers, 3D designers, applied data scientists, and program managers who can build cloud-native solutions and creatively think of ways to map the physical and digital worlds. Additionally, it will be important to support the tool chain with better solutions for integration testing, performance testing, and stress testing of end-to-end solutions.

I believe that the Metaverse in all its evolving forms has the potential to be a significant development across many domains as the 21st Century unfolds. I have attempted in this article to describe several of these, and to suggest potential implications for engineering and data science. I encourage you to follow developments as work on the Metaverse continues to unfold and provides new applications of — and opportunities for — data-driven systems.