Cointime

Download App
iOS & Android

Mixed Reality Is Our Digital Future

Validated Individual Expert

For many people, the word “metaverse” has been tainted with negative connotations, conjuring either cartoonish virtual worlds filled with creepy avatars or opportunistic platforms selling “virtual real estate.” For others, the word metaverse inspires a shrug followed by questions like — “Why would anyone want to spend their time doing that?”

Personally, I believe it’s way too early to write-off the metaverse but we have real work to do. The industry needs to deploy immersive experiences that are more realistic, more artistic, and focus far more on creativity and productivity than on questionable NFT schemes. In addition, we need to overcome the biggest misconception about the metaverse: the flawed notion that our virtual future will replace our physical lives.

This is not how the metaverse will unfold.

Don’t get me wrong, there will be popular metaverse platforms that are fully simulated worlds, but these will be temporary “escapes” that users sink into for a few hours at a time, similar to how we watch movies or play video games today. On the other hand, the real metaverse, the one that will impact our days from the moment we wake to the moment we go to sleep, will not remove us from our physical surroundings.

Instead, the real metaverse will mostly be a Mixed Reality (MR / AR) in which immersive virtual content is seamlessly combined with the physical world around us, expanding and embellishing our daily lives with the power and flexibility of digital content. But don’t take my word for it — the industry is rushing in this direction. Just this month HTC unveiled an impressive new headset (the VIVE XR Elite) that enables powerful mixed reality. And it’s widely reported that Apple will unveil a mixed reality product in June. When talking about this product, Apple CEO Tim Cook said last year — “AR is a profound technology that will affect everything. . . we are really going to look back and think about how we once lived without AR.”

Why Mixed Reality?

Simple. We humans don’t like being cut off from our physical world. Sure, you can give someone a short demo in a fully simulated virtual world, and they will love it. But if you have that same person spend an hour in fully immersive VR, they may start to feel uneasy. Approach two hours, and for many people (myself included), it’s too much.

This phenomenon first struck me back in 1991 when I was a VR researcher at Stanford and NASA, studying how to optimize depth perception in early vision systems. Back then, the technology was crude and uncomfortable, with clunky graphics and lag so bad it could make you feel sick. Because of this, many researchers believed we just needed better hardware, and people wouldn’t feel uneasy. I didn’t quite agree.

Certainly, I knew better hardware would help, but I was pretty sure that something else was going on, at least for me personally — a tension in my brain between the virtual world I could see and the real world I could sense (and feel) all around me. I believed it was this perceptual conflict between two opposing mental models that made me feel uneasy and made the virtual world seem less authentic than it should.

To combat this, I wanted to take the power of VR and combine it with my physical surroundings, creating a single immersive experience in which my visual, spatial and physical senses were all perfectly aligned. My hope was that the mental tension would go away if we could interact with the real and the virtual as if they inhabited the same perceptual reality.

So in 1991, I pitched the Air Force Office of Scientific Research and was funded to develop a prototype mixed reality system at Wright Patterson Air Force Base. It was called the Virtual Fixtures platform, and it didn’t just support sight and sound, but touch and feel (with 3D haptics), adding virtual objects to the physical world that felt so authentic they could help users perform manual tasks with greater speed and dexterity. The hope was that one day this new technology could support a wide range of useful activities, from assisting surgeons during delicate procedures to helping technicians repair satellites in orbit through telerobotic control.

Of course, that first mixed reality prototype at the Air Force didn’t support surgery or satellite repair. It was developed to test whether virtual objects could be added to real-world tasks and enhance human performance. To measure this, I used a simple task of moving metal pegs between metal holes on a large pegboard. I wrote software to create a variety of virtual fixtures that could help you perform the task. The fixtures ranged from virtual surfaces and virtual cones to simulated tracks you could slide the peg along (via 3D haptic models) while optics and early passthrough cameras aligned the activity in 3D. I even used early 3D audio technology developed at the Air Force (AAMRL) to ensure sounds were spatially aligned. And it all worked, enabling greater speed and precision.

Virtual Fixtures project (USAF): user and task board / side by side 1993

I give this background because of the impact it had on me. I can still remember the first time I moved a real peg towards a real hole and a virtual surface automatically turned on. Although simulated, it felt genuine, allowing me to trace along its contour. At that moment, the real world and the virtual world became one reality, a unified mixed reality in which the physical and digital became a single perceptual experience that satisfied all your spatial senses — visual, audio, proprioception, kinesthesia, and haptics. When that was achieved, you stopped thinking about which part was physical and which was simulated — it was just reality.

That was the first time I experienced a true mixed reality. It may have been the first time anyone had. I say that because once you interact with the real and virtual combined into a single immersive experience, all your senses spatially aligned, the two worlds snap together in your mind. It’s almost like one of those visual illusions where there’s a hidden face you can’t see, and then something clicks, and it appears. That’s how a true mixed reality experience should be: a seamless merger of the real and the virtual that’s so natural and authentic that you immediately realize our digital future will not be real or virtual, it will be both — one world, one reality.

The Technology of Mixed Reality 30 years apart (1992 to 2022)

As I look ahead, I’m very impressed by how far the industry has come, particularly in the last few years. The image above (on left) shows me in 1992 at Wright Patterson Air Force Base developing mixed reality (MR / AR). The image on the right shows me in 2022, wearing a Meta Quest Pro headset with color mixed reality capabilities. Over the 30 year span during which my hair went gray, the technology has improved by staggering amounts — in performance, efficiency, and size.

What’s not apparent in the picture above are the numerous full-sized computers that were running to conduct my USAF experiments thirty years ago, or the cameras mounted on the ceiling, or the huge wire harness draped behind me with cables routed to various machines. That’s what makes this new wave of modern headsets so impressive. Everything is self-contained — the computer, the cameras, the display, the trackers. And it’s all comfortable, lightweight, and battery-powered. It’s remarkable.

And it’s just getting started. The invention of mixed reality is an ongoing process, with amazing new products poised to take off. And it’s not just the impressive new headsets from Meta, HTC, and (potentially) Apple that will propel this vision forward, but light weight eyewear and creative software tools from companies like Magic Leap, Snap, Microsoft, Google, Lenovo, Unreal, Unity and many other major players.

At the same time, countless developers are pushing the limits of creativity and artistry, unlocking what’s possible when you mix the real and virtual, from new types of board games (Tilt Five) and powerful medical uses (Mediview XR), to remarkable outdoor experiences from Niantic Labs.

This is why I am confident that the metaverse, the true metaverse, will be an amalgamation of the real and the virtual, so seamlessly combined that users will cease to think about which elements are physical and which are digital. We will simply go about our daily lives and engage a single reality that is magically embellished. It’s been a long time in the making, but 2023 will be the year that our mixed reality future really starts to take shape.

Comments

All Comments

Recommended for you

  • Taiwan’s Ministry of Interior has approved the establishment of a cryptocurrency industry association

    Taiwan's Ministry of the Interior has approved the application of the local cryptocurrency industry to establish an industry association. The local cryptocurrency industry working group, which was established last year to prepare for the establishment of the industry association, said that the working group now needs to complete all preparations and officially establish the cryptocurrency industry association by the end of June as required by the government. The working group is currently composed of 22 cryptocurrency companies, including Taiwan's major exchanges such as MaiCoin and BitoPro. The working group pointed out that ACE Exchange has been expelled from the group because the troubled exchange is under investigation by prosecutors for improper behavior by its former executives.

  • Grayscale ETH Trust negative premium rate is 22.77%

    According to ChainCatcher news and Coinglass data, the Grayscale Bitcoin Trust Fund (GBTC) has a premium rate of 0.02%. The Grayscale ETH Trust has a negative premium rate of 22.77%, and the ETC Trust has a negative premium rate of 36.58%.In addition, the Grayscale BCH Trust has a premium rate of 238.13%, the LTC Trust has a premium rate of 380.60%, the SOL Trust has a premium rate of 515.93%, the MANA Trust has a premium rate of 726.65%, the LINK Trust has a premium rate of 713.66%, and the FIL Trust has a premium rate of 3057.89%.

  • Net inflows into spot Bitcoin ETFs reached $179 million on March 28

    Spot on Chain, a blockchain data monitoring platform, posted on social media that the net inflow of spot bitcoin ETF on March 28th reached 179 million US dollars, a decrease of 26.9% compared to the previous trading day. After 54 trading days, the total net inflow accumulated to 12.13 billion US dollars, which is the level before the last fully negative trading week. BlackRock's iShares Bitcoin ETF (IBIT) and Grayscale's GBTC both saw a significant slowdown in daily inflows and outflows on March 28th.

  • Bitcoin spot ETF had a total net inflow of US$179 million yesterday, and the ETF net asset ratio reached 4.25%

    According to SoSoValue data, the Bitcoin spot ETF had a total net inflow of $179 million yesterday (March 28th, US Eastern Time).Yesterday, Grayscale's ETF GBTC had a net outflow of $104 million, and its historical net outflow is $14.77 billion. The Bitcoin spot ETF with the highest net inflow yesterday was BlackRock's ETF IBIT, with a net inflow of approximately $95.12 million, and its historical total net inflow has reached $13.96 billion. The second is Fidelity's ETF FBTC, with a net inflow of approximately $68.09 million yesterday, and its historical total net inflow has reached $7.56 billion.As of now, the total net asset value of Bitcoin spot ETF is $59.1 billion, and the ETF net asset ratio (market value compared to the total market value of Bitcoin) is 4.25%, with a historical total net inflow of $12.12 billion.

  • Ethereum Inscription ETHS rose over 95% in 24H

    CoinGecko data shows that Ethereum Inscription ETHS has risen by 95.9% in the last 24 hours, now reporting at 7.51 USDT. Earlier, Ethereum founder Vitalik released the latest long article "Ethereum has blobs. Where do we go from here?". As a result of this news, the price of Ethereum Inscription ETHS soared.

  • Binance exec sues Nigeria’s National Security Agency over detention

    According to CoinGape, Tigran Gambaryan, a detained executive of Binance, has filed a lawsuit against the National Security Adviser (NSA) and the Economic and Financial Crimes Commission (EFCC) in Nigeria. Local media reported that on March 28th, Tigran Gambaryan sued the National Security Agency, accusing it of violating his basic human rights and seeking five major remedies from the court.He urged the court to approve the return of his passport and to release him immediately after more than three weeks of detention. He also requested a ban on future detention in similar investigations and demanded public apologies from the National Security Agency and the EFCC.In addition, he requested that the court pay the full amount of compensation for the lawsuit.

  • Vitalik: As L2 transaction costs decrease, there’s no reason why Ethereum can’t be widely adopted

    After the upgrade and introduction of blobs on Ethereum Dencun, Ethereum founder Vitalik Buterin shared his insights on the future direction of Ethereum's expansion. Vitalik emphasized the transformation of Ethereum's expansion from basic expansion work to centralized, progressive enhancement. Vitalik also stated that developers' focus will shift to the application layer. Ethereum will maintain its roadmap centered on L2, and applications will migrate from L1 to L2 to benefit from faster and more cost-effective transactions. An upcoming upgrade is Data Availability Sampling (DAS), which aims to increase the data space for each slot to 16 MB. Progressive expansion improvements include gradually increasing blob capacity, improving data compression, and EIP-7623 (aimed at reducing maximum block size). Vitalik pointed out that with the reduction of L2 transaction costs, there is no reason why Ethereum should not be widely adopted.

  • RWA project Midas completes US$8.75 million in seed round financing

    According to Jinse Finance, RWA project Midas has completed a seed round of financing worth $8.75 million, led by BlockTower, Framework, and HV Capital, with participation from institutions such as Coinbase Ventures, Ledger, GSR, Hack VC, Axelar, and FJ Labs.

  • Two Different Sentences for FTX Founder Sam Bankman-Fried: 25 Years and $11 Billion vs. 16 Months and $8 Billion Losses

    The founder of FTX, Sam Bankman-Fried, has been sentenced to 16 months in prison and charged with eight criminal counts, including money laundering and conspiracy. He was involved in a scheme that caused customers to lose $8 billion and allegedly diverted customer funds to Alameda. Bankman-Fried's lawyers had requested a lighter sentence, but the judge rejected their argument that the collapsed company had vowed to return money to its customers. Prosecutors had sought up to 50 years in prison for Bankman-Fried.

  • Web3's Great Gambit: Incentives for the Almost Impossible

    “When you’re young, you look at television and think, There’s a conspiracy. The networks have conspired to dumb us down. But when you get a little older, you realize that’s not true. The networks are in business to give people exactly what they want. That’s a far more depressing thought. Conspiracy is optimistic! You can shoot the bastards! We can have a revolution! But the networks are really in business to give people what they want. It’s the truth.” - Steve Jobs