Extended Realities and their success
Extended Reality (XR) is quickly becoming a significant trend in research and industrial applications. XR is an umbrella term encompassing technologies such as:
- Augmented Reality (AR): the real world is enriched with some digital info
- Mixed Reality (MR): the real world and virtual world coexist, and mixed interaction is possible
- Virtual Reality (VR): full-immersive virtual environment and interactions
Each of those technologies allows different degrees of simulation of reality in terms of immersion, content, and interactivity levels with the environment. This generates many opportunities for content creators as they can control reality representations and extend them with a whole range of tailored-designed elements and the audience that can enjoy content in new ways and from different remote locations.
This high degree of flexibility of XR results in a huge room for expression, especially from the design and artistic value of an experience. In recent years, this technology has been embedded and utilized in quite a few cultural events and spaces, ranging from museums to outdoor exhibitions to provide more appealing experiences and spark people’s interest in art through technology. Nowadays, due to the Covid-19 pandemic, it becomes even more prominent to deliver content virtually as physical public events attendance is limited or impossible.
There is a wide range of applications for XR today, thanks to the ability to build and immerse users in various environments. These virtual environments replicate or even improve the physical world, resulting in more possibilities and a larger demand for XR. Every year, the user demand for immersive devices expands. Immersiveness is a key feature of digital media in all of its types. This has been established in the tradition of painting, photography, dramatic and installation art, and it is now targeting new heights with transforming realism in XR.
The majority of its current success can so far be directly linked to the gaming industry. However, XR technology can be found and used in a variety of other areas, including medicine, architecture, sport, and the arts. Further on, if in those applications the gamification is used in order to motivate users, those XR applications are called serious games. Such gamification methods could be as well used for increasing users’ motivation to explore different artistic settings.
Therefore, with such a wide industry market, including innovative art exhibitions and acts,it is important to understand and research the possibilities of these new technological solutions.
Perceptions of human participants
Immersion, motivation, and emotions are especially interesting for research projects dealing with user experience. When creating one XR experience, the user's expectations and requirements must be considered. This can be done using human user participants in user tests, where during these test experiments, the behaviour and the perceptions of the user are collected and recorded. This is done with a goal that one can, during the design process of XR experience, estimate immersion and emotions of users for a particular XR experience. As a consequence, it is essential to understand the user and their interaction behaviour for designing a successful XR experience.
Including multiple stakeholders
For example in the project “I like the Digital, but does it like me?” a prototype of an immersive virtual performance environment was developed in a team including stakeholders from different disciplines (development, design, performance art).
The focus during the project was on the appearance of a virtual being and its acoustic representation in augmented reality. It allows a dialogical situation and exchange between the user and the virtual being. The main driving question is how spatial and digital components have to be constituted to encounter a virtual being in an art space. Furthermore, in which relationships different media have to be arranged to create, under the conditions of the actual gallery space, a hybrid performance environment between the real space and virtual space.
Influencing factors
Some of the research projects that we have done have shown how different factors can influence user experience in XR applications. Therefore while creating and designing an e.g. artistic XR experience, one should consider different influencing factors such as human (e.g. Who is going to be visiting your XR artistic exhibition?), system (e.g. What device are you going to use and what feature does this device have?), and context (e. g. Where and when will XR experience take place?).
Depending on these influencing factors, one can set some exciting XR experiences. Some of the studies that we have done, have shown that by using XR applications one can increase motivation for sports, or can affect user’s immersion and emotional state by showing different 360 degree videos. Further on, for XR experiences, an important factor is that users feel like they are present in the told virtual story. Having a good application design, and enhancements such as multiplayer options, made them feel more involved in virtual worlds.
Sharing with others - social co-presents
A fundamental component of artistic experiences is sharing them with others, from friends to strangers. The importance of such aspects naturally shifts to shared XR virtual environments and interactions, where many people are socially co-present. An exciting aspect of multi-user settings is the possibility of artists delivering real-time performance art, getting a more intimate experience with the audience, which can influence the show from different locations.
There are many possibilities to set up such experiences, keeping in mind that relevant design questions on dimensions like people’s virtual representation, communication capabilities among one another, degree of interactivity, social acceptability, etc., must be addressed. For example, in one of our recent studies, we were interested in the influences of interactivity and social environments on user experience and social acceptability of VR. We found out that the degree of interactivity of the experience has an impact on users' experience. Also, users reported that the social environment in which the application is used strongly affects if they find it socially acceptable or not. So, you can ask yourself as well, would you feel comfortable putting on a head-mounted display in a public environment around other people? Or not (yet)?
In the case of artistic experiences with users co-present in a physical environment, special attention has to be paid to the dimension of the shared space and setting of the area that has to be chosen considering the max number of simultaneous users expected and the kind of XR device used.
Exciting Artistic Experiences
Designing exciting artistic experiences with new media types such as extended realities is an exciting new topic. Together with new media types, also new factors for the generation of experiences have to be considered.
This needs to be done including multiple stakeholders in the development process, at least from the technological and artistic field. It is then fundamental to ask human participants how they perceive these created realities. Lastly, during the creation process is as well important to consider influencing factors such as human, system, and context especially in social situations constructs as social co-presents have an important role in creating XR experiences.
Have Fun and game on!
– Dr.-Ing. Jan-Niklas Voigt-Antons, Tanja Kojić and Maurizio Vergari