Unifying Protocols for Conducting Systematic Scoping Reviews with Application to Immersive Learning Research

The progress of immersive learning research as a field requires a clear vision of its status, of the current knowledge being produced and of the open problems and gaps. Typical survey efforts however suffer from lack of systematization, providing a scattered perspective of the field. We have combined the literature on conducting systematic scoping reviews and applied it to the field, presenting the resulting protocol. It contributes a clarification on the sequence of steps and processes for delineating a gap, finding the evidence and depart from it to conduct literature reviews.


INTRODUCTION
A comprehensive, current perspective of immersion learning research is necessary for the field to progress steadily. It can contribute to the development of common language among the research community and a common perspective on the body of literature, so that diverse areas of research more easily and consistently come together. The field of immersive learning research is currently unfocused and scattered due to a lack of common language on its foundational concepts, and consequential scattered perspective on the status of the field.
As we show in the next section, Immersion has a long history from the fields of narrative [1], psychology [2], and mass communications [3] that precedes its current, technocentric focus; Learning research has itself evolved to encompass a varied and context-rich set of areas of interest. The emerging interdisciplinary field of immersive learning research is thus the complex, multidimensional combination of these diverse fields. With such a rich background, it is a bit surprising that the field lacks a clear vision of its broad status, open problems, and gaps. But indeed, a fragmented pattern emerges when analyzing its surveys, as we show in section III. The very concept of immersion is mostly being approached via a split between a psychological state and an objective characteristic of the technical system, with little attention being given to earlier, more prevalent perspectives employed in other scientific fields. Thus, we start by clarifying that the literature already provides a unified approach to the definition of immersion, which if reaching widespread use could shed light on the current, fragmented nature of the field.
Overall, this status has resulted in a lack of systematization across the field, providing a muddled perspective of the current knowledge. Too many past efforts have leapt into systematic literature reviews without previously defining key terms and detailing methods, without identifying where the evidence is, where the reviews should be conducted and how they should focus. Systematic scoping reviews are a powerful epistemological tool to tackle this problem, since they allow researchers to "examine the extent, range, and nature of research activity, determine the value of undertaking a full systematic review, summarize and disseminate research findings, or identify gaps in the existing literature" [4].
As a contribution to change this status or immersive learning research, we synthesized scoping review approaches by Levac et al. [4] and the Joanna Briggs Institute's (JBI) [5] to develop a systematic sequence of steps and processes for the various phases of scoping reviews: delineating a gap; finding the evidence; and progressing from the evidence to conduct literature reviews. Then we instantiated the protocol for immersive learning research, to support its use by the research community in this field. To accomplish this, we established foundational definitions of immersion and immersive learning, by leveraging the literature -we integrated the definition laid out by Agrawal et al. [6] into the unified concept of immersion provided by Nilsson et al. [7]. We have also clarified the concept of immersive learning and employed it to customize the criteria stages of the unified protocol, thus readying it for application to immersive learning research.
The resulting protocol, potentially adaptable for other technical fields, can now be readily applied in order to conduct systematic literature surveys studies in immersive learning research, contributing to continuous progress towards a better understanding of the field and, ultimately, better science in immersive learning research.

A. What is Immersion?
The concept of immersion seems deceptively intuitive, and this illusory simplicity may have contributed to the cursory attention that many papers in the area give to it. As Murray's often-cited work expressed, the term originally referred to being submerged in water, summoning the metaphor of a "sensation of being surrounded by a completely other reality" [8]. This viewpoint is found in the literature of technology-infused fields across two complementary theoretical perspectives: that of immersion as a property of the technological systems that provide this surrounding, developed by Mel Slater and his team [9], and that of immersion as a perceptual response to that surrounding, developed from Witmer & Singer [10]. Subsequent works by a diversity of authors contributed to both perspectives. In fields such as psychology, literary studies or educational sciences, other theoretical perspectives emerged from that metaphor, such as the roles of narratives, engagement, psychological flow and others.
Given that decades have elapsed since Immersion started to be employed as a concept in research, it is surprising that the literature in the field of Immersive Learning Research remains fragmented regarding the definition of immersion. Even literature surveys fail to provide sufficient evidence for their definition of immersion or simply choose a definition ad hoc (see next section). For example, many of the literature reviews merely cite Slater's definition [9] without any explanation or consideration of alternatives. This is unfortunate, as literature reviews that are not grounded in scholarly literature defining immersion cannot effectively contribute to the field. As a result, it may be difficult to use the results of these literature reviews as a means to provide a foundation for future research, and it certainly may explain why a majority of reviews do not cite each other.
However, two recent author teams have sought to remedy this problem. First, Nilsson, Nordahl & Serafin [6] reviewed a variety of perspectives on immersion and created a threedimensional taxonomy conceptualizing immersion: system immersion, narrative immersion, and challenge-based immersion. Rather than provide a single definition for immersion, they instead assume it as the conjunction of these dimensions. This taxonomy, being three-dimensional, can be visualized as a cube, showing how each kind of immersion can be experienced by itself, or in tandem with one or more of the others, to varying degrees, resulting in a spatial positioning within that cube. Their approach defines system immersion as an objectively measurable property of the system and not the product of a user's reaction to that system (as per Slater [9]), and this enables them to subsume a common perspective about immersion, that stemming from Witmer & Singer's seminal view of immersion as a perceptual response [10], which in this taxonomy becomes as a redundant overlap without additional descriptive power. Then they explain narrative immersion "...as characterized by a degree of mental absorption or intense preoccupation with the story, the diegetic space, and the characters inhabiting this space" [10, p. 114]. Finally, they define challenge-based immersion as a user's mental absorption brought about by the experience of challenges requiring mental or sensorimotor skills" [10, p. 116].
Nilsson et al.' perspective lacked a specific definition. This was later expressed by Agrawal, Simon, & Bech [5], who also sought to resolve the disjoint views on immersion as medium vs. experience, as "a phenomenon experienced by an individual when they are in a state of deep mental involvement in which their cognitive processes (with or without sensory stimulation) cause a shift in their attentional state such that one may experience disassociation from the awareness of the physical world" (p. 5). This definition is consistent with the Nilsson et al.'s perspective, providing a clarifying summary of the concept. Thus, for this paper, we use Agrawal's definition and adopt Nilsson's taxonomy as an explanatory metaphor. Now that we have defined immersion, we turn to the combination of the immersion with learning.

B. What is Immersive Learning?
In Learning Sciences, traditionally the main areas of research were focused on how experts differ from novices, the transfer of learning to different contexts and situations, how children learn at different ages, and neuroscience research on the mind and brain. Conversely, teaching-related research tends to center on the design of learning environments, effective teaching examples in different subject areas, teacher learning, and the use of technology to support learning [11]. More recently, holistic approaches have emerged, advocating a perspective on learning as a phenomenon framed by a diversity of contexts (cultural, social, cognitive, and biological) [12], Thus current learning and teaching research now seeks to understand not only learning aspects, processes, and content, but contributions from these wider contexts: motivation, applications of research to school practice, the use of digital technologies for learning, lifelong learning, and more [12]. For instance, the US National Academies of Sciences, Engineering, and Medicine included as part of a research agenda the need to research the influence of learning environments, including aspects such as "how the culture of the learning environment influences learners' sense of belonging, adaptability, agency, and learning outcomes (...) identify the types of learning associated with particular learning tasks and environments and (...) track the predicted consequences for learning, motivation, emotion, and social interaction (...) explain how methods of instruction prime a positive connection between current learning efforts and desired future outcomes" [12].
"Immersive learning" thus emerges as part of this novel research perspective on the relevance of the context for learning and teaching, with immersion providing a theoretical lens to analyze, interpret, and shape that context. For example, researchers interested in learning differences of experts vs. novices may analyze these differences under the lens of immersion, by considering system, narrative, and challenge dimensions. Conversely, other researchers may interpret learning transfer by exploration of its relationship with immersion, via narrative elements (e.g., diegetic space) challenge dynamics, and system context. Still others may be interested in promoting teacher's education, by shaping the system properties, challenge structure, and narrative elements towards different immersion spaces.

C. Clarify your Definitions and Scope
As shown by the subtleties of the concepts provided in the two previous subsections, conducting a scoping review in the field of immersive learning research without prior clarification of definitions can be risky. By simply considering "immersion" to be a technological phenomenon, a scoping review will ignore whole fields of knowledge that may in hindsight be deemed relevant. Conversely, by not delimiting the scope, you begin a data collection effort which is lengthy and wasteful of time and effort, because you do not make important connections between the literature available and the actual objectives for the scoping review. Thus, immersive learning researchers beginning a scoping review should start by reflecting on the definition and dimensions of immersion as well as the wider perspective of learning and teaching research as occurring within an encompassing context. They should then use these delimiters to frame goals, establish clear definitions for concepts driving the scoping review, and ultimately establishing the proper scope.
For example, we are currently conducting a scoping literature review on actual accounts of immersive environments being employed for learning as reported by researchers. Using the definition of immersion above, we realized our goals were aligned with the use of digital technologies, and thus excluded from the scope works about immersion emerging solely from oral storytelling, physical sports, paper books, or mechanical airplane simulators. Subsequently, by considering the narrative and challenge dimensions of immersion, we recognized that digital technologies such as word processors or PDF files would qualify, even though they were not aligned with our goals. This helped us to clarify that our concern was with technology designed with the intent to elicit immersion. Thus, considering the definitions carefully allowed us to much better refine the scope of our work beforehand.
Similarly, by considering the wider perspective on learning research, we avoid accidentally excluding areas that would be relevant for our goal. The analysis of the panorama and diversity of interests in learning research would eventually lead us to identifying the overall area of interest as the application of research to practice, and within that the need to consider as distinct aspects the uses, practices, and strategies with immersive learning environments. Likewise, researchers envisaging a scoping review in this field should consider the definitions and scope involved by framing their interests over this combined perspective of immersion and learning research.

III. CURRENT STATUS OF IMMERSIVE LEARNING RESEARCH SURVEY EFFORTS
We collected a brief methodological sample of literature surveys on immersive learning research by harvesting from Google Scholar papers with two search strings for the date span 2000-2020: 1. Title with: (survey OR review) AND immersive AND learning 2. Title with: (survey OR review) AND learning + Publication name with: immersive OR virtual After correcting citation data on the results and consolidating duplicates, we got n=37 papers. We removed book reviews, books (not chapters), posters, and project reports but not technical reports (n=32), a foreign-language paper which we were unable to read (n=31) and those inaccessible behind paywalls not included in our institutional subscriptions (n=27). We then excluded those not related to immersive learning, or not actually being literature reviews, by reading their titles and abstracts and, when in doubt, the contents (n=12). The resulting corpus is presented in Table I.
By analyzing the concepts of "immersion" employed by these surveys, as shown in Table II, a third (4/12) of the sample neglects to clarify the term, simply employing it. More than half (7/12) address the technological fidelity aspect of the definition. Two address engagement immersion, one addresses interaction immersion, and one addresses narrative immersion. Only a fourth (3/12) of the papers address more than one aspect of immersion.
The theoretical grounding on which the surveys build from these perspectives is thin. A third (4/12) of the papers neglect to provide a reference for the concept of immersion. However, since almost all the remaining papers provided isolated references for their concepts of immersion, we analyzed those to establish their original grounding of the term -that is, we checked the actual theoretical basis for their employed concepts of immersion, either directly in the referenced papers or indirectly in their snowballed references. Fifty-eight percent of the papers (7/12) either neglected to ground the concept of immersion or pointed to references which did not do it themselves. Of the remaining survey papers, three provided a reference which was either directly or indirectly grounded in the concept of technological fidelity immersion, and one paper each provided references directly or indirectly grounded in interaction immersion, engagement immersion or narrative immersion. Also, only one survey in this corpus references an earlier survey from the same corpus. It was expected that a survey would reference earlier survey efforts from its field, but this expectation was not confirmed.
Regarding the use of scoping for establishing the range and location of evidence for the literature reviews, as Table  III details, none of the papers in the corpus employed systematic scoping. A third conducted a scoping stage, planning the literature review with several dimensions of scoping, but two-thirds (9/12) either did minimal scoping (keyword selection or similar) or provided no methodological information on how they conducted their review. Sum of immersion perspective is not n=12 because three papers employ two perspectives.  [4] or the Joanna Briggs Institute's (JBI) proposal [5]. These are summarized side-by-side in Table IV. For clarity, we have homogenized numbering. For instance, Levac at el. use #n for a stage number and then na, nb and nc for its substages. JBI doesn't use any numbering, only different levels of headings. We use for both the nomenclature n., n.1, n.2, etc. We also aligned the stages and substages as much as possible. For instance, Levac et al.'s stage one mostly resembles JBI's second stage, so these appear in the same row of [not mentioned in this protocol]

A. The Unified and Refined Protocol
Both proposals summarized in Table IV provide articulate methods for conducting systematic scoping reviews. JBI's is more detailed, particularly in preparation stages, but on the search and analysis process Levac et al. provide details which are absent from JBI's. For instance, compare JBI's 4.3 with its mirror in Levac et al., 4.2, 4.3 and 4.4. We have combined the details for clarity. However, we have opted not to include the optional stage (consultation) of the Levac et al. process, viewing it as a subsequent and independent study that may stem from the scoping review, rather than part of the review itself.
The interdisciplinary nature of immersive learning research leads to some shortcomings in both protocols, such as situations where prior actions are impacted by later decisions. For instance, JBI's proposes 2.6 (define concept to guide the scope and breadth) after having defined inclusion criteria and types of participants. However, "participants" in immersive learning studies may be not just humans, but also organizations, government bodies, information systems and technology, or yet other foci of study. And those foci can be approached in significantly different ways, such as outcome analysis vs. practice reports. Thus, we recommend defining the concept before establishing the criteria and split these into three clear categories: concept-based, participant-based, and context-based criteria.
Thus, the combined protocol presented in Table V goes beyond merging the two prior ones: it builds on them considering the interdisciplinary and transdisciplinary nature of immersive learning research.

B. The Protocol as Part of a Larger Process
The protocol presented in the previous section is part of a larger process for establishing the current knowledge in immersive learning research: 1) Defining the scope; 2) Locating the evidence; 3) Locating the knowledge (doing several systematic reviews based on the located evidence). The scoping itself may be seen as a sequential process, but as Table V details it includes two iteration moments. Thus, Fig.  1 is provided to highlight this overall and iterative workflow.  3.8 Narrative description of the search decision process accompanied by the search decision flowchart. [5] 4 Extracting and analyzing the results [4,5] 4.1 Develop data extraction protocol with table and/or forms [5] 4.2 Iterative data extraction and extraction protocol refinement Two researchers independently extract into the table/form data from the first five to ten included studies and meet to determine whether data extraction is consistent. Process- [4,5] [4] oriented data may require qualitative content analysis (e.g., thematic analysis) for refining table/form.

4.3
Plan the data analysis process [5] 4.4 Conduct the previously planned analysis (including descriptive numerical summary analysis and qualitative thematic analysis). [4] 4.5 Write a logical and descriptive summary of the results [5] 4.6 Classify the extraction results under main conceptual categories. [5]

Discussion
Consider the meaning of the findings as they relate to the overall study purpose; discuss implications for future research, practice and policy.
[5] [4] 6 Conclusions and implications for research and practice [5] 6.1 Match the review objective/questions and include an overall conclusion based upon the results of the scoping review. [5] 6.2 Provide clear, specific recommendations for future research based on gaps in knowledge identified from the results of the review. These should include suggestions for future systematic reviews that may be of primary interest in view of the outcome of the scoping review.
[5] [4], [ VI. FINAL THOUGHTS This paper synthesized the two main protocols used for systematic scoping reviews into a single, more robust, protocol, potentially applicable to any technical field. Then we customized it for immersive learning researchers, defining the field's foundational concepts of "immersion" and "immersive learning" based on recent literature in the field. Next, we clarified the application of several of the phases and steps, by providing examples or guidelines of aspects that immersive learning researchers should consider, based on the literature. through the FCT -Fundação para a Ciência e a Tecnologia, I.P., as part of project UID/CED/00194/2019, SCReLProg.