The metaverse and its impact on human rights, the rule of law and democracy E-Book

Council of Europe

• Herausgeber: Council of Europe
• Kategorie: Geisteswissenschaft
• Sprache: Englisch

Seitenzahl: 222

Veröffentlichungsjahr: 2024

Beschreibung

How can rights and freedoms be guaranteed in the metaverse, whilst the frontiers between the physical and virtual worlds are becoming increasingly blurred?

This collaborative report by the Council of Europe and the IEEE Standards Association navigates the complexities of technology and human rights, emphasising the importance of a human-centric approach to immersive realities development, such as the metaverse.

The report highlights key issues and risks while exploring the potential benefits of the metaverse. Grounded in ethical considerations, it underscores the necessity of upholding the principles of human rights, the rule of law and democracy.

Authored by a consortium of over 50 IEEE experts and peer reviewed by the Council of Europe relevant sectors, the report provides essential perspectives on technical, ethical and governance dimensions. As the metaverse evolves, so too will the guidance offered within, ensuring policy makers remain informed and adaptable in this dynamic landscape.

Leseprobe

 

The metaverse and its

impact on human rights,

the rule of law

and democracy

 

Contents

Click here to see the whole table of contents, or go on the « Table of contents » option of your eReader.

Acknowledgements

The Council of Europe and IEEE are grateful to the experts who contributed to this report through written submissions or interviews: Prof. Melodena Stephens (Mohammed Bin Rashid School of Government); Monique Morrow; Patricia Shaw (Beyond Reach); Dr Katie Evans; Prof. Irene Kamara (Tilburg University); Dr Mark McGill, Dr Mohamed Khamis and Melvin Abraham (University of Glasgow); Dr Ola Michalec (REPHRAIN Centre, the University of Bristol); Andrés Domínguez Hernández (Alan Turing Institute); Richard Jones (the University of Edinburgh); Prof. Jan Gugenheimer (TU Darmstadt); Cristina Fiani (UKRI); Dr Alicia Cork (University of Bath); Dr Yu Yuan (IEEE SA 2023 President); Jason Douglas Evans, Dr Palak Patel and Dr John Bosco Acot Okello (Nurenyx); Ansgar Koene (EY), Ricardo Chavarriaga (Head of the Switzerland Office of the Confederation of Laboratories for AI Research in Europe (CLAIRE) and IEEE SA IC Chair Neurotechnologies for BMI), Prof. Silvestro Micera (EPFL and Scuola Superiore SantAnna), Dr Kim Barker (University of Lincoln and ObserVAW); Wenqu Chen and Cheng Chi (CAICT); Epaminondas Christophilopoulos (UNESCO Chair on Futures Research, FORTH); Dr Atif Wolfgang Bhatti, Dr Cecilia Drepper, Dr Alexander Andreas, Dr René Döring and Dr David-Julien dos Santos Goncalves (all Linklaters Germany); Karim Mohammadali and Tom Gault (Google); Bugge Holm Hansen (Copenhagen Institute for Future Studies); Cornelia Kutterer (University of Grenoble and Considerati), Prof. Eleni Mangina (University College Dublin); Prof. Ming Li (Utah State University); John Daozhuang (Johnny) Lin (1stCycle Corporation); Prof. Cecilia Metra and Prof. Martin Eugenio Omana (University of Bologna); Alejandro Moledo del Rio (European Disability Forum); Tyler Jaynes (University of Utah); Dr Ramesh Ramadoss (Chair, IEEE Blockchain Technical Community), Peng Yang (Huazhong University of Science and Technology), Zihang Yin (Wuhan Technology and Business University), Carol McDonald (Gneiss Concept), Denia Psarrou and Melina Efstathiou (Eversheds Sutherland). Special thanks go to Monique Morrow, Melodena Stephens, Katie Evans, Prof. Eleni Kosta (Tilburg University) and IEEESA staff Moira Patterson, Sri Chandrasekaran, John Havens and Dr Clara Neppel for their valuable input, review and comments.

The report was led by Irene Kitsara, IEEE SA, edited by Jennie Steinhagen and Irene Kitsara and reviewed by the Council of Europe’s relevant sectors: Media and Internet Governance, Data Protection, Artificial Intelligence, Cybercrime, Democracy and Governance, Anti-discrimination, Youth, Children’s Rights and Sport Values, Education, Human Dignity and Gender Equality.

Enabling technologies and building blocks of the metaverse

An overview of some of the components, the underlying and enabling technologies of the metaverse can facilitate its understanding and related challenges. This overview should not be viewed as definitions or an exhaustive list because there is no alignment in terminology and the technology and technical implementation of the metaverse may vary in the future. It should rather be viewed as working descriptions for the purposes of this report, meant to help in the understanding of the technology making immersive experiences and virtual worlds possible. These components and enablers currently include the following.

Artificial intelligence (AI) and AI systems (AIS)

AI techniques and functional applications, such as machine and reinforcement learning, natural language processing, computer vision and affective computing, enable adaptive and realistic simulations, intelligent virtual entities or agents, customised recommendations and personalised user experiences in general. AI algorithms power virtual characters, natural language processing (NLP) and machine learning optimisation, enhancing the interactivity and responsiveness within the metaverse. By leveraging AI in a responsible and ethical manner, the metaverse can deliver transformative experiences and opportunities for users across a wide range of industries and applications. The recent advancements in generative AI, a subdomain of AI allowing for the generation of synthetic and partially original content, is powered mainly by the deep learning Generative Adversarial Networks, which can generate realistic and dynamic content within the metaverse (pictures, objects, even entire virtual environments) based on existing data and patterns, and the NLP large language models, which allow the understanding and generation of human language. This technology enables the metaverse to continually evolve and expand with new and diverse content, providing users with novel and engaging experiences, including more realistic interactions with AI agents employed in customer service scenarios, social interactions or gaming environments.

Augmented reality (AR)

AR technologies such as AR headsets, smartphone apps, smart glasses or contact lenses overlay digital content onto the physical world, blending the real and virtual worlds, offering a sophisticated on-device sensing, enhanced experience or perception of reality and contextual awareness, or even changing how people, places, adverts and other things look. An AR-powered metaverse is seen as the future of personal computing (replacing the smartphone).

Blockchain

Blockchain technology provides a decentralised and transparent framework for managing digital assets and transactions within the metaverse. Blockchain allows for the creation of unique digital tokens representing virtual assets, such as virtual land, virtual goods or digital collectibles that can be securely owned, traded and verified on the blockchain, ensuring ownership within the metaverse. Blockchain also enables interoperability among different virtual platforms and ecosystems within the metaverse.

Cloud computing

Cloud computing addresses the need in the metaverse for large storage and computing resources, by offering virtual machines. As such, cloud computing is considered to be a critical infrastructure to support immersive experiences, while hybrid clouds, including and combining local and cloud solutions, are under consideration to address concerns related to data security and privacy protection in the cloud.

Digital humans

Digital humans represent a ground-breaking development in XR technology, bringing virtual beings to life with remarkable realism and interactivity. By leveraging advanced computer graphics, animation and artificial intelligence techniques, digital humans replicate human appearance (face or full-body representation), behaviour and even emotions. Digital humans can enhance immersive experiences by serving as virtual avatars (with much more advanced and realistic human representation), guides or companions, providing users with a more engaging and personalised interaction within virtual environments. The rise of digital also raises ethical considerations, such as issues of consent, privacy and the potential for misuse or deception.

Digital twins

Digital twins technology is a powerful tool that enhances XR experiences by creating virtual replicas or representations of physical objects, systems or environments, connecting the virtual and physical worlds, allowing for real-time monitoring, simulation, visualisation and interaction with virtual models that accurately reflect the behaviour, characteristics and status of their real-world counterparts. Digital twins can provide real-time data and insights into immersive experiences, enhancing the sense of immersion, interactivity and realism. Additionally, the combination of digital twins and XR can facilitate collaborative decision making, enabling multiple users to interact with and manipulate virtual representations of physical objects or environments simultaneously.

Edge computing

Edge computing offers a distributed compute architecture which can support the metaverse for low latency and high bandwidth, by processing a large volume of data in a short period of time, for a seamless immersive experience. Edge computing is also expected to reduce the weight and cost of wearables as it will move the currently local compute to a distributed infrastructure, combined with cloud computing.

Extended reality (XR)

XR is another term sometimes used interchangeably with the metaverse that refers to a suite of immersive technologies including virtual, augmented and mixed reality, as well as spatial computing.

Human–machine interface (HMI)

“Human-machine interface [also called user interface or human-computer interface] [is a] means by which humans and computers communicate with each other. The human-machine interface includes the hardware and software that is used to translate user (i.e., human) input into commands and to present results to the user.” (Encyclopaedia Britannica).

Internet of Things (IoT)

IoT technology plays a significant role in enhancing XR experiences by enabling seamless connectivity, data exchange and interaction between physical and virtual elements. In XR, IoT devices and sensors can be utilised to gather real-time data from the physical environment, such as motion, location, temperature and biometric information. These data can then be integrated into virtual environments or used to trigger interactive virtual content. The number of technologies involved is immense; there are over 300 platforms for IoT solutions (Burns, Cosgrove and Doyle 2019).

Mixed reality (MR)

Mixed reality combines the physical and digital worlds, immersing users in a world in which they can interact with digital objects using a combination of eye gaze, hand gestures and voice commands.

Neurotechnology

Neurotechnology is an umbrella term for technologies related to the brain. According to the Organisation for Economic Co-operation and Development (OECD), neurotechnology includes “devices and procedures used to access, monitor, investigate, assess, manipulate, and/or emulate the structure and function of the neural systems of natural persons” (OECD 2019). It comprises brain–computer interface (BCI)/ human–machine interface (HMI), medical implants and neurostimulation. The most relevant type of neurostimulation will be transcranial stimulation, performed with devices that use electrodes on the scalp to provide electrical stimulation to the cortex. Some studies have shown temporary improvement in cognitive capabilities and memory after the stimulation. Some parts of the DIY and gaming communities have shown interest in these systems to improve as cognitive enhancers. There could be a similar uptake for metaverse-mediated applications. Some of these devices are now being released for consumer markets (and can be easily built with off-the-shelf components) (Wexler 2017).

Spatial computing

Spatial computing maps virtual objects into the physical space and allows their integration along with further digital information in the physical environment, enabling a more natural and intuitive interaction and seamless navigation and immersion in the metaverse. Technologies like spatial audio complement the experience and allow a more realistic and immersive experience.

Virtual reality (VR)

VR technology provides users with a simulated environment that can replicate the physical world or imaginary settings. Virtual online worlds are predominantly experienced through either two-dimensional displays (smartphones, monitors) or immersive VR headsets. By wearing a VR headset, users can experience a three-dimensional virtual world and interact with objects and other users. In immersive VR, users experience increasing degrees of presence, body ownership (the illusion they are in that environment and “own” their virtual avatar body) and increasing degrees of perceptual realism (the multisensory fidelity of the experience, including visual, auditory, haptic and olfactory realism). Immersive VR experiences can mimic social experiences in reality, with a consequent degree of psychological realism.

Web3

The third iteration of the internet is focused on a decentralised infrastructure (such as blockchain) which brings openness and decentralisation and empowers users, with decentralised development, control and ownership shared among users and the community. In this phase of the internet, AI systems start doing seemingly intelligent things autonomously (Markoff 2006).

Chapter 1 Introduction and scope

The Council of Europe’s Digital Agenda 2022-2025 points to the metaverse as a development that raises multiple and complex challenges, similar to those experienced with previous technology advances and disruptions like the internet, social platforms or artificial intelligence (AI). Yet the intensity and effects of the metaverse are only expected to multiply and increase. The lack of consensus about definitions and the polarisation of stakeholder opinions about the expected impact of the metaverse resemble the concerns that have emerged with AI in recent years, which range from enthusiasm to scepticism observed in the pattern of AI “winters and summers” hype cycles. Concerns about the legal implications of the metaverse likewise echo the discussions at the time of the internet’s emergence in the late 1990s, as well as during the rise of gaming platforms and virtual worlds. As a cross-cutting environment with possible applications across various industries and all aspects of life, spilling across generations with a significant indirect impact on the planet, areas for consideration by policy makers span almost the full range of human rights and fundamental freedoms.

Since its signing in 1950, the European Convention on Human Rights (hereinafter “the Convention”) has progressed and broadened in scope through the case law of the European Court of Human Rights ( “the Court”). The Court regularly expands and deepens the rights afforded by the Convention, referred to as a living document, and considers their application in new contexts and circumstances not originally conceived by its drafters. In its guide to human rights for internet users based on the Convention and its interpretation by the Court, it is unequivocally stated that “fundamental freedoms and human rights apply equally online and offline”. These frameworks are complemented by what are known as “Council of Europe standards”: encompassing conventions, recommendations, guidelines and best practices; addressing specific issues; and setting out frameworks, rules and principles to be adopted and reflected in national legislative frameworks of its member states. A non-exhaustive list of existing relevant legal frameworks and standards, both international and regional, can be found in Appendix II, while specific references are included in the respective sections of the report. Some frameworks, resources and experiences from other jurisdictions and technology areas are sometimes mentioned as background; they reference different approaches and rationales, while the focus remains on the mandate and perspective of the Council of Europe. Some of these references and considerations are included in the tables referring to specific issues in Appendix III and should be read in combination with the respective sections in the body of the report, bearing in mind that they are illustrative and not exhaustive. The question that emerges is whether the current frameworks, applicable to offline and online reality, remain appropriate or sufficient to address current and future risks and threats to human rights, the rule of law and democracy in the metaverse.

The future composition of a virtual, immersive society, which includes virtual governments, marketplaces, etc., as well as the relationship and impact of this virtual world on the physical world and offline life remain unclear. Accordingly, both the current and the anticipated effects of engaging in the metaverse – known and novel – require active and timely attention. As with other technology disruptions, such as generative AI, the clemency of a long reaction time will not be granted. There is therefore an immediate need for policy makers and governing bodies to: 1. develop a baseline understanding of the technologies and concepts associated with the metaverse; 2. acknowledge the urgency of assessing the current situation and how it may evolve over time; 3. understand macro technological, economic, environmental and social contexts; 4. evaluate the scope, risks and opportunities concerning existing or missing safeguards (legal frameworks, standards and challenges with enforcement and self-governance); and 5. prioritise and enable the uncompromised exercise of human rights and fundamental freedoms to attain human prosperity and social well-being in any and all democratic environments – in the virtual realm just as much as in the non-virtual.

This report provides an overview of the principal issues identified jointly by the Council of Europe and the IEEE Standards Association, a global standard-setting organisation within the IEEE, within the framework of the Digital Partnership. The report aims to support the Council of Europe member states in their understanding of the metaverse and its potential, its applications and benefits, as well as the issues and risks that may arise from the development, deployment and engagement within the metaverse. It also looks at the impact on human rights, the rule of law and democracy – to be further analysed and assessed in the context of the Council of Europe’s work so that policy may be applied and directed accordingly. While not exhaustive, it is grounded in a shared belief that technology, even when complex and still under development like the metaverse, can and should be human-centric, include ethical considerations and aspire to respect human rights, the rule of law and democracy by design (Nemitz 2018). With the knowledge that the metaverse may or may not develop in the way currently imagined, the highlighted issues may evolve in magnitude and importance. For the report, the IEEE brought together nearly 50 experts to share their perspectives and expertise on the technical, ethical, social, legal, policy, regulatory, standardisation and governance issues associated with the environment and applications of the metaverse. Related considerations for navigating within this shifting landscape are offered.

Understanding the metaverse and its current state

Currently there is no single and commonly accepted definition or understanding of the metaverse, although standardisation efforts to harmonise the language and related terminology are underway. The metaverse is often used interchangeably with terms such as virtual worlds, immersive realities, digital twins, virtual, augmented, mixed or extended reality (VR/AR/MR/XR) or Web3; other times it is viewed in a hierarchical relationship to those, either as an umbrella term or as a subset of these terms (Hupont Torres et al. 2023). Terms that are currently used at the European Commission level include virtual worlds – next generation digital worlds as per the ECJoint Research Centre (Hupont Torres et al. 2023) – and immersive realities, while other international initiatives and partnerships continue to refer to the metaverse (WEFMetaverse Initiative; IEEEMetaverse Congress and Metaverse Standardization Committee; ITU Forum on Embracing the Metaverse 2023).

The report does not aim to provide a definition but rather a description of the metaverse, a term coined in 1992 by Neal Stephenson in his science fiction novel, Snow Crash, to describe an immersive virtual world. As per Matthew Ball, author of The Metaverse and How it Will Revolutionise Everything, the metaverse can be described as a vision for a scaled, interoperable network of real-time rendered 3D virtual worlds and environments that can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence and with continuity of data (Ball 2021). From that perspective, the metaverse can be viewed as “the envisioned end state – incorporating all digital worlds alongside the physical world, with interoperability between them all” (McKinsey 2022a). As such, these can be viewed as components/spaces of a single virtual universe – a metaverse with interconnected/interoperable elements, including virtual worlds and gaming platforms.

While much concerning the eventual materialisation of the metaverse remains uncertain, the complex concept it represents is no longer considered science fiction. Virtual worlds and immersive realities are already realities simpliciter. Over the past 30 years, explosive innovations in the enabling technologies of the metaverse – such as breakthroughs in augmented reality and gains in the autonomy of smart systems – have fuelled this rapid transition to our current situation. Related platforms and experiences offer glimpses of the metaverse’s potential, but they are still fragmented and lack the seamless integration and interoperability required for a fully realised metaverse.

According to Merriam-Webster, “meta” means “after” in Greek and “verse” is an abbreviation of the “universe”, “so ‘metaverse’ neatly implies a world or conception that requires the ‘real’ world in order to move beyond it and acknowledge another realm” (Merriam-Webster, “What is the Metaverse?”). As the borders between physical and virtual worlds may increasingly be blurring, the current understanding that the “real world” is the physical world will not necessarily be as easily distinguishable or obvious. The metaverse, conceived to be immersive, creates a virtual and digital extension of the present universe.

It should be noted that some consider that there are or will be several “metaverses” as opposed to the single “metaverse” vision. In the context of the IEEE’s discussions, for example, metaverse refers to an experience in which the outside world is perceived by the users (human or non-human) as being a universe that is built upon digital technologies as a different universe ( “virtual reality”), a digital extension of our current universe ( “augmented reality”) or a digital counterpart of our current universe ( “digital twin”) (IEEE SA 2023).

What makes the metaverse are its features:

► the immersiveness of the experience (with varying degrees, such as 2D versus a full sensorial experience);

► the element of presence (the illusion that the environment you are in is plausibly reality);

► persistence (the virtual worlds continue to exist even when you are not online);

► the convergence of the physical with the virtual world and the effects of one world on the other;

► the interconnectedness and interoperability of the different virtual spaces.

The metaverse brings together and integrates various existing and emerging technologies – related to hardware and software – that provide the architecture and infrastructure needed for the metaverse to function, along with the underlying and enabling technologies that enable immersive experiences and further features of the metaverse. These technologies include, among others: 5G/6G networks that allow data transmission and connectivity; AI systems (AIS); digital twins (a digital or virtual copy of a physical system allowing for simulations and modelling); the Internet of Things (IoT – connecting different devices in the physical world and allowing their seamless connection to the virtual world); blockchain (an infrastructure using cryptography techniques allowing, for example, transactions of physical or digital/virtual assets, typically using cryptocurrencies); augmented reality (AR, which overlays information to the physical world either adding onto or hiding parts of the physical world); virtual reality (VR – providing through the use of devices like VR headsets an immersive experience separate from the physical environment); mixed reality (MR – combining elements of AR and VR); extended reality (XR– referring to ways humans interact with, experience and visually interpret the physical environment through a digital interface (IEEESA 2022a) and encompassing technologies like AR, VR or spatial computing); and brain–computer/human–machine interfaces (BCI, HMI – a means of communication between humans and computers and a translation of user input into machine-readable commands). When used together or in new ways, these technologies create new applications and experiences. In the different future scenarios of the metaverse its scope and impact on offline life and the physical world vary, as do the potential threats and risks to the exercise of human rights and fundamental freedoms, the rule of law and democracy. These technologies are described in more detail in the section “Enabling technologies and building blocks of the metaverse”.

It is important to bear in mind that the enabling technologies may change over time (for example with the use of brain–computer or human–machine interfaces) and they should be viewed as a means to implementation, while their technical specifications and features may be linked to specific benefits, risks and mitigation possibilities. In that sense, the metaverse is technology agnostic and involves the seamless transfer of data, in real-time, between the virtual and physical worlds (Agile Nations 2023). Whether and how the vision for the metaverse will materialise and develop in the future will depend on several factors like adoption, technology development, access to data, regulation and geopolitics. Implementation and mass adoption of the metaverse are linked to some technical requirements and challenges which still need to be addressed. First, it requires great computing power, since existing interactive equipment demands a large computing load and high power consumption. Immersive and compelling end-user experiences, with low consumer and enterprise hardware price points, and low software application development costs are needed to drive end-user adoption of XR technology for the metaverse to be successful. Developments in the enabling metaverse technologies such as AI are expected to bring improvements in the performance, accessibility and user experience of the metaverse and newer headsets are then expected to supplant reliance on physical smartphones and monitors, while heralding new capabilities in augmented intelligence (Zheng at al. 2017), perception (Schraffenberger and Van der Heide 2014; Hugues, Fuchs and Nannipieri 2011; Schraffenberger 2018), embodied communication (Artanim 2020), productivity (McGill et al. 2020a), accessibility (McGill et al. 2020b) and more.

While avatars already exist (a virtual representation of self), digital humans represent a ground-breaking development in XR technology, bringing virtual beings to life with remarkable realism and interactivity. These photorealistic 3D human models have the potential to revolutionise various aspects of XR, including entertainment, education, communication and customer service. Digital humans can enhance immersive experiences by serving as virtual humans in different roles, like virtual assistants, guides or companions, providing users with a more engaging and personalised interaction within virtual environments. They also introduce new avenues for storytelling, allowing users to interact with virtual characters in ways that were previously unimaginable and creating in this way a new type of (social) interaction with AI-controlled agents. The rise of digital humans also raises ethical considerations, such as issues of identity, consent, privacy and the potential for misuse or deception.

Application areas

The metaverse, like the internet and artificial intelligence, is transversal, with different applications covering all aspects of life. It can be linked to goods and services (retail, gaming, social platforms, media) as part of the consumer metaverse; education and research; industry, manufacturing and engineering as part of the industrial metaverse; health, justice, e-government and political participation.

Currently many of the use cases are driven from the gaming industry, with existing big communities of users; however, there is a lot of emerging cross-pollination between various sectors, with industry-specific sectors such as automotive and manufacturing expanding to gaming, for instance. The industrial metaverse is an increasingly important application environment of the metaverse (MITTechnology Review Insights 2023, WEF 2023b). It involves a new industry model and operation system based on the core infrastructure and application concept of the metaverse that serves the industrial economy. The significance and purpose of the industrial metaverse is to generate practical value for specific industrial applications, representing through digital twins, for example, exact physical systems, while the consumer metaverse is based on a user experience with a sense of surrealism. The industrial metaverse will fully rely on digital identity, blockchain and other enablers to build a new generation of industrial systems, while it is expected to bring many benefits, with probably more immediate deployment and scaling up than the consumer metaverse (Nokia 2023).

The metaverse is poised to transform the field of medicine by introducing innovative approaches to diagnosis, treatment and patient care, as well as medical training and telemedicine. Surgeries, potentially involving brain stimulation, will be experienced in new and immersive ways. Of course, the benefits will need to be considered alongside any associated risks concerning patient rights, privacy and mental autonomy. The balance of the benefits and risks will also come under consideration in national and defence environments. In the United States, “metaverse-related ideas are already part of some of the latest military systems. The high-tech helmet for the new F-35 fighter jet, for instance, includes an augmented reality display that shows telemetry data and target information on top of video footage from around the aircraft” (Knight 2022).