The Metaverse
Note: This essay relies heavily on 3 sources:
The Metaverse by Matthew Ball
The Dawn of the New Everything by Jaron Lanier
The Pew Research Center’s Report on The Metaverse
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Origin Story
It’s a tricky thing tracing histories.
How far back do you go? Did the internet start with Mosaic the first browser? Or with Tim-Bernards Lee who came up with the first protocols or Babbage the inventor of the first programmable computer.
In tracing the origin story of the metaverse, we could go back to Ivan Sutherland who invented the first head mounted display in the 1960’s ..or the stereoscope first developed in the 1830s
But for our purposes it comes down to two people. Jaron Lanier and Neal Stephenson. These two represent the two sides of the Metaverse, the ideal and the technology.
Jaron Lanier is now known as the father of Virtual Reality. He co-founded VPL Research, a company that built the first VR headset in 1984. And with that headset came the first experience of virtual reality as we know it today..and with that a new technological, philosophical and (maybe) existential frontier. During that time a literacy scene grew in concert with V.R and the burgeoning internet..called cyberpunk. Themes were usually dystopian, characters were snarky, manipulative, somewhat shallow but undeniably cool (thanks to the tech). Everything hinged on existential doom. A favourite from this canon was Neal Stephenson’s Snowcrash (others were William Gibson’s Neuromancer and Vernor Vinge’s True Names)
In Snowcrash, Stephenson has the honour of being the first to coin the term ‘Metaverse’ (the book was published in 1992). He also was the first to define avatar as a term for a virtual representation of self in the Metaverse (the word avatar has its origins in Sanskrit). In the novel, the Metaverse is a ‘place’, a virtual place where characters can plug in using their headsets..and turn into gods. The protagonist in the novel (whose name is Hiro Protagonist) is a pizza-delivery boy in the real world. In the Metaverse he is a ninja — a sword fighting assassin hell bent on saving the world and taking down the bad guys. In the novel Stephenson doesn’t define what the Metaverse is other than a ‘place’ his characters inhibit once they plug in. The term, however, stuck around. As Lanier explains in an interview, at that time there was a cycle where the technologists were building the first multi-person virtual worlds, Stephenson and Gibson were publishing the early cyberpunk novels, which in turn had a huge impact on cinema (Matrix, Inception and later Ready Player one) which had a huge impact on the cultural zeitgeist and the cultural perception of what a virtual world could look like. Oh and Snowcrash repeatedly comes up as a favourite noel of Zukerberg, Bezos, Musk et al
The Metaverse Today
The term Metaverse seems to be ubiquitous. The top technology companies in the world (Meta, Microsoft, Google) are pouring hundreds of billions of dollars into what’s being touted as the next big computing and networking platform. The shift is being likened to our move from 90’s era fixed line internet PC’s to the era of mobile and cloud computing we live in today.
So what is the Metaverse? There are a plethora of definitions and a lot of confusion as to what it is..even among the tech community. Is augmented reality part of it? Do you need a VR headset? Is it decentralised? I.e would individuals have control over their data and virtual goods etc? Like the ‘the internet’ can there be only one Metaverse or multiple metaverses. The way most people imagine it to be is a fantastic world where you can be a ninja, play chess with a goblin, race through a metropolis while being chased by king kong..oh and do all the boring stuff like attend meetings and educate yourself etc.
Thinking of the Metaverse as a virtual world, the definition that is most relevant and succinct is the one by Matthew Ball,
He defines it as:
“A massively scaled and interoperable network of real-time rendered 3D virtual worlds that can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence, and with continuity of data, such as identity, history, entitlements, objects, communications, and payments.”
Hmmm ok, let’s unpack that
3D Virtual World: A 3D virtual world is any computer generated simulated environment e.g any video game of the past couple of decades or films like Toy Story. Virtual worlds come in many dimensions. For the Metaverse ‘3D’ is critical, otherwise we are describing our current state. 3D is also a natural evolution of online communities and experiences over the past decades. From text based in the 80’s and 90’s, through pictures in the 2000’s and now a near constant stream of high resolution photos and videos. What this progression implies is that humans want to express themselves online, not in a static way, but with a sense of being alive and ‘real’. From photos to videos to 3D could be the next logical step. This will be enabled by technical ability and new social apps.
Real-Time Rendered: Rendering in a sense is solving a maths problem..or an equation. An equation whose inputs are capacities of graphics processing units (GPUs) and CPU’s (Central Processing Units) and other data (e.g. rules to determine when and how) and output a 2D or 3D object. This is a hugely complex procedure that usually takes a lot of time and processing power. For example, for your average Marvel show it takes approximately 4–24 hours to render a single frame. Pixar has a huge ‘render farm’ i.e a supercomputer made up of 2000 machines. This is one of the significant challenges for the imagined Metaverse. For a virtual world to be alive and respond in real time would require a rendering rate of 30–120 frames per second..computing power that currently way beyond the reach of an average user.
Interoperable Network: Imagine you are a happy citizen going about your business in the Metaverse. You spend a good chunk of your time gaming, socialising, attending a work meeting etc there. Ideally if you bought a virtual hat in Minecraft, you might want to wear it while gaming in Roblox..or at your next meeting. Or you might want to resell it on a third-party platform. The Metaverse should enable a person to take, carry and use their assets be it their history, achievements, finances and virtual stuff as they see fit
The term ‘interoperable’ refers to the ability for computer systems or software to exchange and make use of information sent from one another. With our current trusty internet we have the Internet Protocol Suite (TCP/IP), a set of communication protocols that tell disparate networks how data should be packaged and moved around. This suite is managed by the not-for-profit open standards group Internet Engineering Taskforce (IETF) and is the primary reason we refer to the internet as opposed to an internet. Every computer globally from a massive government supercomputer to a 6 year old mucking around on their ipad has embraced the internet protocol suite. Other not for profits were established to make sure no matter how big or decentralised the internet would become it would continue to interoperate. Also important were the establishment of common standards for files e.g JPEG for digital images. Thanks to these standards a photo taken with your iphone can be uploaded to facebook, downloaded to your gdrive and then uploaded again to Pinterest.
The internet as we currently (in 2023) know it demonstrates a near miraculous coming together of systems, technical standards and conventions across an almost dizzying plethora of network devices, operating systems, applications, languages and countries. This is because of the internet’s origin story in government research labs and universities. These institutions wanted to build a ‘network of networks’ and not-for-profit organisations helped build out that vision. Nobody at the time was thinking about the commercial possibilities of the internet
That is not true of ‘The Metaverse’
The Metaverse in its current form is built by corporations for the sole purpose of commerce, data collection and the sale of virtual goods. All of the popular virtual worlds today use their own proprietary rendering engines, file formats and have no systems through which they share data with other virtual worlds. As a result, existing virtual worlds have no clear way to find and recognise one another nor do they have a common language in which they can coherently communicate. The key roadblock is establishing standards. Establishing standards is a very human problem masquerading as a technological problem. They are established through negotiation, compromise and consensus, not discovery..unlike the laws of physics. As of this writing the big players aren’t willing to make those concessions.
The driving force of interoperability, however, will not be technology but economics. The Metaverse economy will depend on attracting more users and more developers, which in turn will lead to better experiences, which in turn would make it more profitable to operate, driving greater investment. To leverage the metaverse economy is to adopt common standards. Those who adopt them will have an advantage.
Massively Scaled: It’s in the name: ‘Meta’ from the Greek roughly translates to ‘which transcends’ and verse as in universe. So the Metaverse is intended to be the unifying layer that sits above and across all computer generated virtual worlds.
To get a little granular and maybe Russian-doll this: The Metaverse would contain meta-galaxies, which in turn would contain various virtual worlds. For example Roblox is a meta-galaxy with a network of millions of virtual worlds (e.g the game Adopt Me). Or Facebook. Facebook is not the internet but a 2D galaxy of sorts i.e a tightly integrated collection of pages and profiles.
The Metaverse, like the internet, or our universe will have to contain all these ‘galaxies’ and a ‘massively scaled’ number or virtual worlds and provide the underlying infrastructure (like gravity) to hold it all in place.
Persistence: To Reset or Not!
The philosopher John Locke tried to explain personal identity as a continuity of memory or consciousness. In Stephenson’s vision of the Metaverse, characters inhabit a richly detailed virtual world, where they have conversations, get into fights, visit places and everything they ever do persists forever.
Not so much in our current virtual worlds. In games such as Fortnite or World of Warcraft maps or territory end, enemies respawn, activities repeat players don’t have a choice of whether an event endures indefinitely. The only thinking that persists is the player’s memory and their record of having defeated an enemy or bought an item. This is less of a problem for virtual world games, but if we are to replicate other facets of life in the Metaverse such as education or work, we would need actions, achievements and interactions to reliably endure.
Synchronous: The vision for the Metaverse is not one of static experiences but one of shared experiences. According to Matthew Ball, synchronous experiences in the Metaverse will be one of the hardest problems to solve. The internet was not designed for synchronous experiences but the sharing of static copies of messages and files from one party to another. And our current online experiences have hoodwinked us in a way. What feels continuous and synchronous, really isn’t. For example, because of how data travels from various servers and networks to your computer, Netflix is pushing content to you before it’s needed, should there be a temporary delivery error. The problem is latency, i.e the data receives an instruction to move but for some reason it takes its time and causes a delay (usually microseconds). Currently it’s most acute (though seldom noticed, or when it is, it really is!) in video conferencing software. Usually different members on the call receive ‘live’ video and audio a quarter, or half a second behind or ahead of one another. However the software sends you packets you missed, speeds up playback and edits out pauses so at least in appearances you are in sync with everyone else.
Virtual worlds with all its participants would have much higher performance requirements. Far more complex data sets are being transmitted to multiple users, who would need that data continuously..so a loss of transmission to any one individual (no matter how miniscule) affects the entire collective experience.
The Games People Play
The imagined Metaverse isn’t simply a hop skip and a jump from the internet we know and love. We will need new cabling infrastructure, wireless standards, hardware equipment and potential overhauls to the foundational elements of the Internet Protocol Suite. Overhauls that would require opt-in from thousands of different internet service providers, private networks, router manufacturers and more. Even a substantial update is unlikely to be sufficient for a globally scaled metaverse.
The enormity of the challenge is likely to alter the balance of power between tech giants, developers, users etc. There seems to be a winner takes all mentality amongst the big players, who seem to be prepositioning themselves in expectation of the Metaverse..even though its arrival is still in the distant horizon.
However, as I write this in 2023..like Mario defeating Bowser and saving Princess Peach the come-from-behind underdogs and likely pioneers of the metaverse will be the video game industry. Those pesky kids (and a very large proportion of adults) getting sucked into the worlds of Fortnite, Roblox, World of Warcraft are the trailblazers…well those actually building those platforms are.
Video games are fun, and that ‘fun’ has compounded for close to seven decades and is now worth close to $180B. The companies that are typically focused on gaming and powering video game consoles are some of the most powerful tech companies in history (so when I said underdogs..) Companies that a few have heard of..Nvidia anyone? Well it’s one of the top ten largest public companies in the world. Its founder Jensen Huang has said video games occupy a rare niche where the market is ginormous and it demands really powerful computers that are small in size (as opposed to academia and defence where the market for powerful computers is tiny). Games are responsible for driving down the cost of 3D graphics hardware that has made virtual reality possible. Similarly with software. The best real time 3D rendering software comes from gaming. And they have a head start in battling the internet’s archaic infrastructure. They have required synchronous and continuous networking connections since the mid 90s and had to invent technologies such as predictive AI (that takes over during a network drop) to compensate. And hey video game companies are the gold standard experts in creating worlds that people actually want to spend time in.
The (imagined) Future that’s already here?
Are we there yet?
There’s imagination and then there’s reality. The metaverse will only become ‘The Metaverse’ if it can support a large number of users experiencing the same event, at the same time, and at the same place, without a whole bunch of concessions in user functionality, interactivity, latency, rendering quality etc etc etc. However, things are happening. Even the biggest proponents of the Metaverse such as Zukerberg and Epic Games’ Tim Sweeney admit the realisation is far off but are nevertheless committing to making it a (virtual) reality. As mentioned above, the economic incentives could prove strong enough to smash through any hurdles. Technology is improving exponentially in concert with the proliferation of affordable mobile devices which are equipped with CPUs and GPUs capable of powering and rendering real-time rendered environments. Gaming companies are investing heavily in ‘cross-platform gaming’ i.e the ability for gamers to play one another even if they use different operating systems. They could buy virtual goods through any platform and use them on another, and carry their in-game history across platforms.
And then there’s COVID..
For a long time making a fake avatar and designing your ‘house’ and tending to your ‘garden’ (e.g. in Second Life) was maybe considered…a little weird. Virtual worlds were in the domain of other loner hobbies like recreating the battle of Gettysburg with tiny figurines..or knitting (I look forward to your letters) not to mention virtual weddings and funerals (which apparently were happening since the 90s). What COVID did was kick the door down and accelerate the process of de-stigmatisation of time spent in virtual worlds. The impact of two years inside was huge. Millions of sceptics are now ready participants in virtual worlds such as Animal Crossing and Fortnite. Everybody has now attended events online that were once planned for the real world. The average gamer has now ballooned across demographics to bring in more than just 13–34 year old men. Even before COVID from its launch in 2017 through to 2021 Fornite generated an estimated revenue of $20 billion. Even more amazingly Fortnite is now one of the largest sellers of ‘fashion’, out-grossing Dolce & Gabbana, Prada and Balenciaga by multiples.
By the end of 2021, Ford, Nike, Louis Vitton, Marvel and Justin Beiber have all made the metaverse a key part of their business and their growth strategy.
Proto Metaverse: Uses and Abuses
A lesson from COVID was that ‘Zoom — school’ was awful. Students lacked the ability to interact, go on field trips, build robots, dissect a frog etc etc. Pedagogical research shows the importance of tactility when it comes to education. At home or distance education is a poor substitute for in person education, but also in person education is often a luxury and depending on circumstances not for everybody. Technologies such as volumetric display, VR and AR headsets, haptics and eye-tracking cameras and rich virtual simulations could be the middle ground. The ‘killer app’ for VR in education was thought to be the ability to ‘visit Rome’. That proved to be boring but students could learn how to build Rome in a day (maybe a semester), learn how aqueducts work by constructing them or take part in a Roman senatorial debate. The benefits of this is that these experiences do not have a marginal cost, thus making a quality education more equitable. This does not suggest that education in the Metaverse will be easy. Teaching is an art. Learning is hard to measure. But VR has the potential to enhance learning, expand access and reduce cost. It could also deliver exponentially greater reach for wonderful teachers and their work.
Like most technologies in the computing era which had their start in gaming and consumer leisure and then made their way into industry and enterprise applications, the Metaverse could be the same. In construction for example Unreal-based real-time simulations are currently being used to visualise large construction projects in 3D e.g. in Tampa Florida. Through this simulation it was possible to see how the city would be affected by the construction process, how it would affect local traffic, how these buildings shape emergency response times in the area. In the future, these simulations won’t be passive but active. Signage could be altered based on real time tracking e.g a starbucks could change signs based on the type of customers that use the store. The ‘simulation’ could connect into the city’s underlying infrastructure systems, enabling AI powered traffic lights to operate with more accurate information and so help city services to better respond to emergencies.
In medical use, in 2021 neurosurgeons at John Hopkins performed the hospital’s first ever AR-surgery on a live patient. According to the doctor who led the surgery, “it’s like having a GPS navigator in front of your eyes in a natural way so you don’t have to look at a separate screen to see your patient’s CT scan” Comparing surgery to with AR to driving a car with GPS is possibly where the rubber hits the road (you got to allow me one pun). Currently for gamers, VR games aren’t good enough to compete with consoles (or the PC or smartphone for that matter) but with surgery the question is would AR/VR have a meaningful impact on the outcome? A car doesn’t need GPS, the trip will be made regardless. However a GPS does shave off time and (in my case) anxiety. If AR could afford surgery a higher success rate, faster recovery rate or lower cost, this could justify their adoption and thus could be the ‘killer app’ for the Metaverse.
There are many players in the proto-metaverse but the big 5 Google, Apple, Facebook, Amazon and Microsoft (GAFAM) will have a disproportionate impact. They are making direct strides such as Oculus’ foray into education, however GAFAM are more powerful than their direct influence or revenues suggest. These will have an impact on all of the revenues they don’t recognise (eg Amazon’s data centres, Google ads and Apple’s 30% cut on app revenues) because they own the platforms and are able to set technical standards and influence business models. However disruption is not a linear process. Investment and conviction do not ensure success. Microsoft in the 90s had a head start when it came to operating systems, devices and resources but failed to read what customers wanted and Google, Apple, Facebook and Amazon beat them by providing better software and tech.. We have already mentioned the ‘upstarts’ in the gaming industry such as Epic, Unity and Roblox. They might not have the clout of GAFAM but they have the player networks, the developer networks, the virtual worlds and the ‘virtual plumbing’ to be the real leaders of the Metaverse.
And then there’s all that data…
Metaverse technologies offer enormous varieties of application from education to construction and of course entertainment. There will no doubt be other uses that we haven’t thought of but what we can probably assess from how technology progresses these will offer more personalization, greater cognitive amplification, emotionally intensive immersions and extreme escapism. The technology will continue to advance and become more accessible to larger segments of the population, and people will surely leverage it to work more intelligently.. or amuse themselves into mindlessness.
The other cruel lesson especially from social media is that technology does not solve human problems like bias, fear or violence. They accentuate, accelerate and amplify what is already present within us. So if the metaverse as a virtual world becomes a reality we could see even more isolation, more concrete echo chambers and further dissociation from our communities.
“Unfortunately, without comprehensive efforts in developing net literacies in the general population for cognitive immunity and in developing security and privacy best practices, it is unlikely that humans will yet have overcome all of the current problems of fragmented attention, distraction, digital security, privacy and persuasive fake media, so these phenomena will still exist in possibly frightening and powerful new forms.” — Mike Liebhold, Institute of the Future
In Stephenson’s metaverse characters were judged by the quality of their avatars. A black and white avatar meant it was someone of low importance and could not enter the exclusive ‘areas’ of the metaverse. Tech can be polarising and exclusionary. Some will greatly benefit, the creatives and those who unlock the new business models. However as with the advent of any new technology there is an excitement and push forward..ahem..’move fast and break things’..that could engender hostility, discrimination and exploitation to marginal and vulnerable groups of people. And on this trajectory everybody will have their data exploited and monetized. There is also the potential for the metaverse to be a place for the privileged as summarised in the quote below
“Things that in ‘real’ reality are free to see (architecture, clothing design, street art) can in a metaverse system be paywalled off, such that only people with the right token can see the ‘true’ shape of the building or the full details of the dress. The nightmare iteration of this is that essentially everything has to be paid for to experience — it’s a world where everything is an NFT.
— Jamais Cascio, distinguished fellow at the Institute for the Future
Conclusion
So far we have addressed the metaverse as a ‘virtual place’. However, what if the metaverse isn’t a virtual place but a moment in time. Then the transition would be seamless..or pernicious. As technologist and entrepreneur Shaan Puri writes in this twitter thread the metaverse isn’t a virtual place but a moment in time when our digital lives will be worth more to us than our physical life. Our attention used to be wholly (close to 100%) in our physical environment. TV dropped that to 85%, computers to 70%, phones ~ 50%. We have gone from valuing friends to valuing followers, playing physical sports to gaming, valuing real tangible assets to digital assets (NFTs and as mentioned Fortnite is one largest sellers of fashion in the world, out grossing many fashion labels). Projecting this trend 10–20 years, it is very feasible for a company to make smart glasses that could sit in front of our eyes all day (google tried and failed ). That could lead to ~90% of our attention in the digital world. And where our attention goes, our energy flows (as Shaan Puri has said). That would be the true dawning of the ‘metaverse’. And if humanity approached that crossroads of living life ‘virtually’ or reality my hope is that we choose reality. Stephenson’s novel was clear: In Snowcrash the ‘metaverse’ made life in the real world worse
In closing the below quote from Jaron Lanier (the father of VR) provides an apt way to move forward
“My peak optimism was probably in the 80s when I was introducing the concept of virtual reality to the world. And what I saw in virtual reality was potentially something that would be wonderfully good for the world. And I still think it could be and maybe will be. I saw a way to finally appreciate how wondrous our given reality is by having a point of comparison. We take what we have for granted so easily. But if you’ve spent some time in virtual reality and then you go into a real forest, I think you’re able to love that forest in a more visceral way than is readily apparent otherwise. And many, many similar examples. Just looking at someone else’s face is astonishing after you’ve been in virtual reality for a while”
