Researchers Use ‘League of Legends’ to Gain Insights Into Mental Models

Psychology researchers have used the game League of Legends to advance our understanding of how people build “mental models” – the mental tools that allow people to make use of complex systems. The researchers found that mental models are developed in three distinct stages – findings that have the potential to inform everything from workplace training to video-game development.

“We create mental models to help us interact with systems, whether that’s driving a car or playing a video game,” says Caleb Furlough, lead author of a paper on the study who did the work while a Ph.D. student at North Carolina State University. “These models allow us to predict how the system will react when we perform different actions. For example, if I turn the steering wheel to the right, the car will turn right.

“Our goal with this study was to get a deeper understanding of how people structure these mental models in their memories, and how those structures change as we gain expertise,” says Furlough, who is now a human factors consultant. “In other words, how does the structure of a mental model differ between a novice, a journeyman and an expert?”

“We chose a video game for this study because it presented a new arena for mental model research, and because it requires people to use different types of memory – not just retaining facts, but also memory related to skills and actions,” says Doug Gillan, a professor of psychology at NC State and co-author of the paper.

For this study, researchers first recruited several people who played League of Legends. This small group served as a focus group, which the researchers used to develop an in-depth survey that could be used to capture a player’s understanding of various concepts in the game, such as strategies, characters and locations.

Researchers then recruited 158 study participants with varying levels of experience in playing League of Legends. Novices were classified as those who had played the game for 100 hours or less; journeymen were classified as those who had play the game for between 101 and 1,000 hours; experts were classified as those who had played the game for more than 1,000 hours. All of the study participants completed the survey, which focused on the extent to which key game concepts were related.

The researchers then analyzed survey responses using the Pathfinder analysis tool to create visual networks that highlight the connections between various concepts. The researchers then compared the networks of novices, journeymen and experts in order to identify differences in how each mental model was organized.

The researchers found that novices organized their mental models based largely on “natural language connections,” or the way that words are used in everyday conversation – as opposed to the way language may be used in the context of the game.

In contrast, journeymen mental models were based largely on procedural connections. This means their models connected concepts based on how they could be used to perform actions or accomplish goals within the game.

Expert mental models displayed even more pronounced procedural connections, but also demonstrated two other characteristics: abstract connections and subnetworks.

Abstract connections rely on theoretical links, rather than obvious, external characteristics. For example, if an ambush relies on splitting one’s forces, the link between the concepts of “ambush” and “divided forces” is an abstract one.

“And subnetworks are little groups of highly interconnected concepts,” Furlough says. “We only see a lot of subnetworks once someone has achieved a level of expertise, granting them significant insights into the interconnection of myriad concepts.”

These insights into how people build mental models have potential applications for developing training programs in general, and possibly for game design in particular.

“For training, this work offers insights into which characteristics a training program may want to focus on at each stage of the process,” Furlough says. “For example, you may want to start with a focus on language, then move to procedural connections before introducing abstract connections. And, if subnetworks are relevant to a given skillset, a training program may want to teach the relevant concepts together as part of a single package.

“As for gaming, these three stages of mental model development could offer a distinct approach to game development – with games introducing challenges that force players to change their mental models as they gain in expertise.”

The paper, “Mental Models: Structural Differences and the Role of Experience,” is published in the Journal of Cognitive Engineering and Decision Making.

-shipman-

Note to Editors: The study abstract follows.

“Mental Models: Structural Differences and the Role of Experience”

Authors: Caleb S. Furlough and Douglas J. Gillan, North Carolina State University

Published: May 7, Journal of Cognitive Engineering and Decision Making

DOI: 10.1177/1555343418773236

Abstract: Mental models are mental representations of the external world which humans constantly use when they interact with the environment and systems within it. These mental models are in part constituted by an underlying structure of associated concepts which are modified as a person gains experience with a system or domain. Video games provide a context that encourages the development of sophisticated mental models. The current research sought to understand how mental model structures differ between video game players of varying experience levels. Participants were recruited both over internet forums and through Mechanical Turk. Mental model structures were measured using relatedness ratings between pairs of concepts that were derived from players with high levels of experience playing League of Legends. Relatedness ratings were transformed into Pathfinder networks which were used to analyze mental model structures. Results revealed structural differences in mental models between experience levels. A three-stage model of mental model structure development is proposed to explain the results which suggests that some structural characteristics appear earlier in mental model development than others. The role of mental model structural characteristics is discussed in light of both the design of training programs and video games.