Exploring educational immersive videogames : an empirical study with a 3D multimodal interaction prototype

Gestural interaction devices emerged and originated various studies on multimodal human-computer interaction to improve user experience. However, there is a knowledge gap regarding the use of these devices to enhance learning. We present an exploratory study which analysed the user experience with a multimodal immersive videogame prototype, based on a Portuguese historical/cultural episode. Evaluation tests took place in high school environments and public videogaming events. Two users would be present simultaneously in the same virtual reality environment: one as the helmsman aboard Vasco da Gama’s XV-century Portuguese ship, another as the mythical Adamastor stone giant at the Cape of Good Hope. The helmsman player wore a virtual reality headset to explore the environment, whereas the giant player used body motion to control the giant, and observed results on a screen, with no headset. This allowed a preliminary characterization of user experience, identifying challenges and potential use of these devices in multi-user virtual learning contexts. We also discuss the combined use of such devices, towards future development of similar systems, and its implications on learning improvement through multimodal human-computer interaction.