How VR/AR Help to Achieve Immersive Experiences in Gaming

Currently, there is a common culture of innovation and technological advances. Technology is being incorporated into all sectors of our daily life. Also, many technical terms and words are starting to appear in our vocabulary although, many times, they describe concepts that have been in use for some time; such is the case of “Serious Games”.

Many researchers talk about the future of Serious Games including the advantages of using them for training and learning purposes and, also, how we can improve the results obtained using Serious Games. A key point is that the experience is perceived as truly real and, to achieve this, truly immersive systems are needed.

But…

What Are Serious Games?

The term “Serious Game” is sometimes confused with other related terms such as “Simulator” or “Virtual Reality”. Depending on the environment used, the difference between them can be very subtle, but it is important to clarify each one in this context.

On the one hand, a simulator is an application that replicates a certain situation, natural phenomenon, process, or experience. These kinds of developments can be very different types but the majority are computer-based simulations and normally have a very important visual component to represent the result of the simulated entity.

On the other hand, Virtual Reality (VR) creates a completely unusual experience, offering to the users a synthetic and alternative world so that users believe they are actually living this virtual experience. It is possible to consider VR as a specific kind of computer-based simulation because the main objective of this kind of application is to simulate, inside the virtual settings, the perception cues that a person would perceive in a real situation so as to make the users believe they are experiencing a real situation when they are inside a virtual one.

Between these kinds of applications, we can find gaming. The principal objective of games is entertainment and for that they use several kinds of technologies and applications. Some games are simulators but most of them use virtual reality. So, where can we place the term Serious Games?. This term comes from the evolution of games and video games to cover some kind of learning or training necessities and the development of specific skills in a comfortable, enjoyable and relaxing environment.

In particular, one of the most popular definitions is:

“A mental contest, played with a computer in accordance with specific rules, that uses entertainment, to further government or corporate training, education, health, public policy and strategic communication objectives” given by Michael Zyda in 2005 in his paper “From visual simulation to virtual reality to games”

In other words, Serious Games (SGs) are computer-based games whose main mission is NOT ENTERTAINMENT and fun but are developed for training and learning purposes or for improving certain user skills. Other terms related with this concept include “educative games”, “educational games” or “edutainment”, which are less formal and technical than SGs but are used in a wide variety of fields.

These kinds of applications do not need to simulate a particular phenomenon, and if they do, the simulation’s accuracy is normally less than in a simulator. For their development, different technologies can be used, among them Virtual Reality (VR) and Augmented Reality (AR) or a combination of both. These elements are more likely a component in an SG or an interaction paradigm used to accomplish the educational objectives of the game design.

SGs are considered a very successful environment to provide awareness about different aspects of life and improve learning skills. There are several researches suggesting that these applications have an important impact in generating public interest and an increase in awareness[1] and also the benefits and advantages of them for teaching concepts, addressing specific problems or training certain skills[2]. These capabilities are due to how attractive these applications are for all types of individuals: both appealing to young people due to its ludic nature, and to adults as they provide rigorous information about diverse issues in a relaxed environment.

But What About Engagement?

In general, “engagement” is considered as a generic indicator of user involvement in the system or game used, so it can be an important indicator of how interesting and attractive that application may be. High levels of engagement can positively impact the user’s attitude towards the learning tool, his/her emotional state while using it and the desire to play it again to continue learning. Here immersive technologies are proven to obtain a better engagement than non-immersive technologies.

The usage of virtual and augmented reality is well-known in today’s gaming industry. They are very important interaction points and very useful in getting games successfully launched to a wide variety of the general public. The technology is more present in our lives and the gaming industry always tries to bring the latest technological trends to an audience eager to try and experience new things. VR and AR have a very important presence in the gaming and video areas, and their incorporation in non-entertainment uses are increasing every year.

For that reason, the inclusion of technological components that enhance 6DOF (six degrees of freedom) immersion by motion tracking in Serious Games could influence in a significant increase of its engagement. In this sense, the VR and AR are good candidates to achieve an engage serious game, where the user is learning, training or improving certain skills while he/she is playing and enjoying, using a very attractive technology.

Immersive Experience

Serious Games can only be a choice if the technology evolves into real immersive solutions. In June of 2017, we had a full package of information coming from Google I/O 2017 #GoogleIO2017, a breath of fresh air into the future of VR. Goggle takes VR very seriously and, after having sold millions of cardboard goggles, they announced they are now betting on the fully immersive experience by boosting Daydream with new hardware. They also announced a collaboration with HTC Vibe for new stand-alone headsets. Every component of VR, the CPU and the GPU is built directly in the headset. These innovations give the user several grades of freedom:

1. No need for High End expensive PC’s
2. Wireless → You can move
3. No need for a mobile phone → No dependency on the manufacturers or mobile devices.
4. Standardization in VR apps may boom. Now it is painful for so many players
5. Inside out tracking -> No need for cameras or sensors for hand tracking
6. True to scale positional tracking
7. Precise 6DoF
8. Sensors integrated into standalone VR thus reducing the floating point imprecision and increase precise tracking.

According to Deloitte’s, immersive learning is one of the four key applications of VR/AR and the learning experience is maximized when there is full and precise 6DOF motion tracking of the own body and instruments.

Advantages of VR

Virtual Reality environments have been used in learning simulators for years. They were not exactly originally conceived as games and their usage was more oriented to the adult public. Its evolution and the appearance of new immersive technologies, such as the Samsung Gear VR or Oculus Rift (vincular a páginas?) and other low cost solutions make VR more accessible to the public in general. It also makes possible the development of a wide variety of applications based on VR and with many different purposes, from leisure to educational, and for all ages.

There are many advantages of using this technology in multiple applications but, in Serious Games, significant educational developments and advantages include the following:

• It makes possible the adaptation of each environment to the specific needs and profile of the user or player. This aspect is very important in learning, because not all students have the same level of learning or the same learning speed.
• It is highly interactive, making the applications more attractive, enjoyable and engaging.
• Provides a very important capability to reproduce almost every kind of scenario using virtual models.
• Has the possibility to simulate situations of any kind, increasing the learning areas where it can be utilized.
• It is possible to control the learning process with user identification, score registration and evaluation of the individual’s performance.
• Allows the control of the stimulation level offered in each tool.
• It is possible to create a predictable and repeatable environment where the situations can always happen in the same way. This is a very important key in learning: the possibility of repetition of the same situation or scenarios which are very difficult to completely reproduce in real life. This aspect can be very useful in cases of users with special needs.
• Learning or training can be accomplished in a very safe environment so it will be a perfect technology in case of considerable risky training situation or learning concepts.
• There are a wide variety of technological devices that can be incorporated to improve the user feelings, engagement and immersion in the application. For instance, the use of motion tracking sensors to control the position of user, his/her hands or specific elements which motion must be controlled to produce a very important sense of presence, are possible in this kind of applications.

Advantages of AR

If VR seems to be a powerful development tool for training or teaching, what happens if the information comes to you in a real world environment? The possibilities of Augmented Reality applications are incredible because the knowledge can be offered in the space where the learning or training needs occur.

The real environment is augmented with information which can help the user learn or train for specific skills. In this case, AR is supporting the real world rather than replacing it with virtual environments. Its everyday usage is more often in games, tourism, leisure, training and education. The main advantages of AR in educational environments or Serious Games are[3]:

• It is possible to offer all the components needed to learn concepts or train for specific skills in the scenario where the person has to develop his/her work skills later.
• Offers a safe environment of learning and training, avoiding the risk that a real world experiment may entail.
• It is possible to break the limitations of space and time so students can observe the inner workings of simulated elements by entering inside them (particles, atoms, etc). Also, long experiments which can take extended periods of time can be simulated via AR in a substantially shorter amount of time.
• Allows the virtualization of historical people, celebrities or scientists, bringing them to the real world around the student, and enabling interaction with them to understand their story, their experiments or their contribution to humanity.
• It is possible to show works of art, monuments or other types of cultural heritage symbols directly to students, inside their current environment, allowing them to move around, investigate and to receive information about them and for analysis and discussion with other students and teachers.
• New technologies including tablets, smartphones or AR glasses make more accessible the usage of this kind of interaction and applications to every area of our daily life. See our post on VR and New Technology Trends for more details.
• The engagements in AR development are quite high since they offer a very natural and intuitive method to acquire knowledge. See our post on applications on Surgery.
• Allows the usage of motion tracking systems, similar to the ones used in VR, to control the user motion and the motion of complementary elements needed to interact with the augmented environment.

If Engagement Is Related To Immersive Experience. What’s Next?

Multiple tracking methods are technologically available as highlighted in the next frame.

Nevertheless, Electromagnetic sensors have several clear advantages as per below figure:

All the advantages and opportunities provided by these technologies, specifically the full immersive experience, make Serious Games a very powerful area where the learning, training and knowledge can be acquired in a very engaged, enjoyable, attractive and highly interactive environment.

References:

[1] Rebolledo-Mendez, G., Avramides, K., de Freitas, S., & Memarzia, K. (2009). Societal impact of a serious game on raising public awareness: the case of FloodSim. Paper presented at the Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games.

[2] Griffiths, MD (2002b). The educational benefits of videogames. Education and Health, 20(3), 47–51. More information.

[3] Y.Li, ”Augmented Reality for remote education”, 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), Chengdu, China, V 3, 2010, pp. 187–191. More information.