Games for Learning at the 2010 Games for Change Festival from info g4li on Vimeo.

Educational game development is increasing in popularity and is tapping into a huge, lucrative market, yet educational game design is sometimes more a matter of intuition rather than based on tested design patterns. A reliable, universal evaluation tool to assess educational games’ designs across genres and content areas is not available, though several researchers and designers are trying to identify core design patterns (Holopainen, 2003; Church, 1999) while others provide rubrics around very limited, design elements (CSUS, 2007; SDSU, 2009).

This presentation encourages educators, researchers and designers to evaluate games with a shared vocabulary and professional dialogue about clear criteria and standards by using a rubric created by researchers from the Games for Learning Institute (G4LI). G4LI is a consortium of university researchers, supported primarily by Microsoft Research, studying the educational use of digital games and investigating their socio-cultural, cognitive and emotional impact (http://www.seriousgamessource.com/item.php?story=20553). Part of this effort is to identify supportable design patterns for effective educational games that designers can draw on to assure high quality gameplay and educational value. A rubric with the dual purpose of providing guidance for educational game design as well as an evaluation tool for comparing educational games across play genres is one outcome of this work.

A rubric is several things: a guide for evaluating performance (Danielson, 1997); a descriptive scoring scheme developed to guide the analysis of a product or process (Brookhart, 1999); a written-down version of criteria, with all scoring described and defined (Arter & McTighe, 2001). While rubrics are commonly used in education, creating a rubric for evaluating game design has been challenging, as fixed criteria about what to assess in a game is not agreed upon. Thus, G4LI researchers surveyed the existing literature and identified 17 “design patterns” of effective games. The rubric uses these design patterns as criteria for game design and evaluation.

Along with the game design patterns, three central criteria are used in the rubric. These relate to internal game mechanics that affect gameplay and learning, and are simultaneously important and independent of one another (Danielson, 1997). Each of the 17 individual design patterns are evaluated on 5-point scale in each of the three criteria:

1. Technical Implementation: The activity of programming and executing a design pattern into a working version of the game. Includes the seamless integration of design elements within game play.
2. Educational Appropriateness: The ability of the game to address educational/curricular goals and the player(s) knowledge/ability relative to the educational content being addressed.
3. Overall Integration with goals: The integration of the design pattern being considered with the other elements within the game, and within overall game play and educational goals.

The rubric has been developed and tested as a guidance-for-design tool in the national G4Li Game Design Challenge, already announced (http://g4li.org/archives/680). Authors will report the effectiveness of using the Game Design Patterns rubric, lessons learned from this experience and, by including a sample game evaluation, demonstrate how the rubric is used.

View this abstract on the GLS 6.0. ]]> http://g4li.org/archives/2306/feed 0 http://g4li.org/archives/2280 http://g4li.org/archives/2280#comments Wed, 26 May 2010 04:35:36 +0000 Charles Hendee http://g4li.org/?p=2280 Microtalks – Design Innovation

• Classroom Multiplayer Presential Games
  Miguel Nussbaum, Pontificia Universidad Católica de Chile
  The popularity of massively multiplayer online games (MMOGs) has grown enormously, with communities of players reaching into the millions. The games’ potential for education stems from the fact that players are immersed in a virtual world where they have the opportunity to manipulate and explore, thus motivating the construction of knowledge. The interaction and collaboration between participants allows students to exchange information, test their understanding, and reflect on what they have learned. Since the number of students in a classroom is not massive and play takes place within a single room rather than on the Internet, we have changed the terms “Massively” and “Online” to “Classroom” and “Presential” respectively, thus giving us the new designation “Classroom Multiplayer Presential Game” (CMPG). We present different types of CMPGs for diverse schools subjects and show design rules. The systems are built with several kids (up to 10) sharing a projected image and each with her personal input device, or by constructing an Augmented Reality world with each child with her personal Tablet.
• Classroom Constraints and the “pass-back” effect: Games Designed to Transcend Generational Divides
  Marjee Chmiel, The JASON Project, and Nina Walia, PBS
  

  Games for learning often have an explicit audience in school-aged children, but there is another audience as well: parents and teachers. While much of the conversation about games for learning focuses on “kid cognition,” scalability can only be achieved when designers acknowledge and design for the parents and teachers who serve as the gatekeepers of the technologies. This microtalk will address design principles when the classroom is the unit of analysis and the “pass-back effect” — when parents hand their mobile device to kids in the backseat or whenever they’re on-the-go — and how this creates unique challenges to optimize kid-friendly, educational mobile apps within the constraints of devices designed for grown-ups. Games from PBS and National Geographic’s The JASON Project will be used as case studies.

• SURGE: Integrating Vygotsky’s Spontaneous and Instructed Concepts Within the Curriculum
  Douglas B. Clark, Mario Martinez-Garza, Vanderbilt University
  Brian C. Nelson, Kent Slack2, & Cynthia A. D’Angelo, Arizona State UniversityIn Thought and Language, Vygotsky discusses the potential for leveraging intuitive understandings from everyday experience (“spontaneous concepts”) with “instructed” scientific concepts to build robust understandings. The question remains whether or not the intuitive spontaneous concepts players develop in through computer games can be successfully leveraged into robust instructed concepts in a manner that transfers to academic assessments and across domains recognized as central by the scientific disciplines themselves. This talk presents data from early studies with the SURGE video game, where students demonstrated significant learning across multiple items based on the Force Concept Inventory that mirror their embodied experience in the game, but also demonstrate difficulties in integrating ideas from the game on items less directly connected to their embodied experience.
• Tobi Sulnier, First Playable

• Victoria Van Voorhis, Second Avenue Software
  

Moderator: Ken Perlin, NYU/G4LI

• –Translating Instructional Ideas into Good Game Designs
  Cornelia Brunner, Center for Children & Technology/Education Development Center, Inc.
  The presentation will examine the design schema we are currently using to investigate ways to use specific affordances of the handheld digital game medium to support progressive pedagogical practices in middle school science teaching. Instructional design determines the interplay between the pedagogical, gaming and science learning requirements for a game.

  We are focusing on integration of digital games into the regular curriculum rather than on replacing the more traditional curriculum with new media learning objects. We are not trying to teach content (e.g., the chemistry of photosynthesis) nor provide opportunities to practice important skills (e.g., observation, classification, etc.) but rather to address recalcitrant misconceptions by providing a shared, actively constructed visual metaphor for the counter-intuitive aspects of a complex idea may help create a conceptual readiness to learn on the part of students that teachers can refer to in crafting their regular lessons.

  We expect the teachers (rather than the game) to provide explanations and engage their students in reflections on the meaning and import of the process depicted in the game. The contribution of the game is to provide a model that is both accurate and age-appropriate and can be manipulated.

• Towards a Theory of Game-based Assessment
  Kurt Squire, University of Wisconsin-Madison
  

• Sources of Evidence for Embedded Assessment in Immersive Game Worlds
  Brian Nelson, Arizona State University
  Whatever theories, models, or approaches may be utilized for assessing the skills and processes identified as important to successful learning in the 21st century, the nuts and bolts of assessment will still consist of interactions between the person or group being assessed and the outside world. However, the media used for interactions between learner and world, the “work products” (Mislevy, Steinberg, & Almond, 2003) through which learners will demonstrate construction of knowledge and skills, will be starkly different from those that existed in the past. In most classroom-based instruction, assessment via multiple choice tests, surveys, and other traditional means continues to be the norm. But such decontextualized tools and activities are nowhere to be found in the immersive game worlds in which 21st century learners spend much of their time outside the classroom. Instead, in modern game environments players are continuously assessed as they embark on long-term learning quests using situated, need-based functions and toolsets through which they demonstrate a trajectory of growing competence in domain specific areas, usually as part of a large community of practice.If immersive game worlds are going to find a place within formal educational settings, researchers need to pay careful attention to the affordances, weaknesses, and design of such environments for learning and assessment. In this session, I take a designer’s look at how data collection activities for learning and assessment are structured in immersive game worlds. I define and describe the Global Evidence Channels through which learning and assessment activities are derived in game world, offering examples of how multiple evidence channels in operation through game based activities can be utilized to construct rich data trails for assessment. Participants in this session will take away heuristics for designing game-embedded learning and assessment activities that take advantage of the elements that such environments support.

Respondents: Tracy Fullerton, USC & Eric Zimmerman

• Playing the Field Methodologically
  Constance Steinkuehler, University of Wisconsin-Madison—Traditional views about research methods would insist that researchers select their methodology to be consistent with their existing beliefs about the nature of the world and our knowledge about it. For many of us, however, the process goes the opposite direction: We tend to construct theory to match the data we collect rather than collecting data to test the theory we already have. Research on new phenomena like “games and learning” are, in practice, the wild wild west of methodology. I argue that, when you’re trying to figure out what the phenomenon is in the first place, it’s perfectly okay to date around with methods before you marry them and settle down. As long as your theory and methods in each study are coherent, why not go ahead and play the field? I use examples from my own research to illustrate how the long-standing distinctions between quantitative and qualitative, objective and critical, might sometimes be usefully flouted.

• Back to Sesame Street: Reviving developmentally-focused formative research
  Katie McMillan Culp, Center for Children & Technology/Education Development Center, Inc.Formative research is a specific approach to guiding the development of media for children that places the focus on the comprehensibility of the media and the meaning-making of the child. It was created to inform the development of educational television programming in the late 1960s (Connell & Palmer, 1971), including Sesame Street and The Electric Company. The production of these shows was deeply and consistently guided by input from developmental psychologists who did regular, sustained testing of show concepts with children within the target age range.

  Formative research can function as an alternative or a complement to research focused on the identification and validation of design principles to guide the development of effective games for learning. Research seeking to identify design principles assumes meaningful connections between those patterns of behavior captured in the research process and correlated outcomes, and prioritizes the codification of recommendations for design. This contrasts with formative research’s emphasis on children’s interpretive relationships with the media they encounter.

  Formative research also provides a rich context for negotiation among the multiple expert perspectives that inform the development of games for learning, by placing the empirical examination of the needs and perspectives of the player at the center of the design and development process.

• Instrumenting games for assessment: What can online processes tell us about what students know and don’t know?Greg Chung, UCLA/CRESST/CATS
  

  Games developed for learning purposes provide an opportunity to observe learning processes as they occur. However, measuring learning processes moment-to-moment requires a way of instrumenting the game such that the data collected yields meaningful information about a student’s understanding. This talk will present our current work on instrumenting a prototype game for teaching the concept of fractional units. Data from a recent study with 150 students using our prototype game generated nearly 250,000 records of event data. This data will be used to illustrate how we instrumented the game, provide examples of the kinds of measures that were extracted from the event data, and show how those measures related to students’ learning processes and outcomes.

• A Framework for Research on Motivation and Game-based Learning
  Richard Wainess, UCLA/CRESST/CATS—Many educators believe the motivational characteristics of games make them effective tools for learning; this is based on an assumption that motivation guarantees learning. Much of this belief about games and motivation dates back to the 1980s when Malone and colleagues cited a number of constructs as providing motivational appeal within games, which subsequently has been equated to learning from games. However, empirical evidence hasn’t fully supported this premise. While motivation may be a contributor to learning from games, and while some of the constructs defined by Malone and others may contribute to learning, research strongly suggests it is the instructional methods embedded in games that determines whether learning occurs. Certain of the proposed motivational constructs represent, or are related to, effective instructional methods and, therefore, can aid in learning. However, other motivational constructs have, over the past 20 years, been shown to potentially counteract effective instruction and can likely be detrimental to learning. Therefore, one cannot simply say motivation is good for learning. It depends on which motivational characteristics are being discussed and how they are implemented. The purpose of this paper is to introduce a model of motivation and learning, based on a large body of empirical data, as a framework to guide future empirical research on motivation and game-based learning. The model is composed of a set of independent variables (motivational constructs), a number of mediating and moderating variables (expectancy-value theory, several key components of cognitive load theory, and several game-relevant instructional methods) and one dependent variable (learning outcomes).

• Research on the Instructional Effectiveness of Computer Games
  Sigmund Tobias, SUNY, and J. Dexter Fletcher, Institute of Defense Analysis— This panel will discuss research on the effectiveness of using computer games for instruction. Sigmund Tobias and J. Dexter Fletcher will update a continuing review of empirical studies examining computer games for the delivery of instruction; the studies reviewed to date were conducted from 1992-2009 and have been posted at https://sites.google.com/a/adlnet.gov/references/. The presentation will emphasize research-based recommendations for game design; slides of the recommendations will be reproduced for the convenience of attendees. The reviewers have monitored and analyzed empirical, data-based literature on the effectiveness of games for instruction over the last four years. There is some confidence in the recommendations since they have remained fairly constant over the project’s duration.
  

• Action video game playing as a learning tool
  Daphne Bavelier, University of Rochester—Although the brain is far from being fixed, the types of experience that promote learning and brain plasticity are still poorly understood. Surprisingly, the very act of playing action video games appear to lead to widespread enhancements in vision, attention and possibly cognition. Such wide transfer of learning is unusual. A common mechanism may be at the source of this wide range of skill improvement. In this view, action video game players are no superhero, but they would benefit by having learned to learn.
  

E19.2500 Video Games and Play in Education
Introduces theories of learning, learning through play, and the role of technology in education. Students will encounter a wide variety of game genres through critiquing and playtesting current and historic videogames. Students will do preliminary game design with history, theory, learning outcomes and learner characteristics in mind.

E19.2176 Designing Simulations and Games for Learning (Plass, Fall 2010)
Introduces a model of designing simulations and games for learning, covering cognitive factors, affective factors, as well as narrative. Examines the potential of various genres of simulations and games as learning technologies and discusses issues of designing for fun versus designing for learning through readings, discussion, play, design and research.  Class discussions focus on identifying and applying design patterns for the design of effective learning games. Students will collaborate to design several learning games and implement paper and digital prototypes.

E19.2510 Narrative, Digital Media and Learning
Addresses the role of narrative when designing serious games, simulations, social media, and documentary storytelling. Narrative forms have been used for teaching and learning given their role in memory, cognition, the engagement of learners, as well as in case studies for learning, teaching, and research. This course explores the design principles and constitutive elements of narrative-centered learning. Special emphasis is given to designing media narratives that enable and support pedagogical models including story-based learning, digital storytelling, and entertainment education, and goal-based scenarios.

E19.2520 Research on Simulations and Games for Learning (Plass, Spring 2011)
Provides an introduction to research on simulations and games, with a focus on choosing the appropriate approach, e.g., playtesting, evaluation, or efficacy research, and the appropriate methods, e.g., think aloud protocols, video research, eye tracking, EEG/EMG, user log data, or biometrics. Reading assignments, class discussions, and case studies will be used to discuss the goals, methods, design, and setup of these methods and prepare students to design and execute their own playtesting and evaluation research for learning games of their choice.

Cognitive Load Theory

Cognitive load theory (CLT) is one of the most important theories in educational technology, a highly effective guide for the design of multimedia and other learning materials. This edited volume brings together the most prolific researchers from around the world who study various aspects of cognitive load to discuss its current theoretical as well as practical issues. The book is divided into three parts. The first part describes the theoretical foundations and assumptions of CLT, the second discusses the empirical findings about the application of CLT to the design of learning environments, and the third part concludes the book with discussions and suggestions for new directions for future research. It aims to become the standard handbook in CLT for researchers and graduate students in psychology, education, and educational technology.

Position Summary:

Administer the operations and outreach activities of the NYU-Games for Learning Institute (GLI). Promote and implement high profile special events; seek funding sources and develop grant proposals; Liaise with schools, collaborators within the university and outside organizations. Coordinate high-level meetings and coordinate communication among collaborators. Manage research assistants. Manage student involvement within the Institute. Prepares monthly, quarterly, and annual reports to G4LI funders.

Official Posting

If you are interested, please follow the link above to the official posting.