Intelligent Science Stations Bring AI Tutoring into the Physical World

You can find our book chapter here. This chapter was published in the new Artificial Intelligence in STEM Education book.

Intelligent Tutoring Systems have been shown to be effective for education. However, so far they have been limited to flat screen computer interactions. We introduce a new genre of Intelligent Science Stations that bring intelligent tutoring into the physical world. Intelligent Science Stations are mixed-reality systems that bridge the physical and virtual worlds to improve children's inquiry-based STEM learning. Automated reactive guidance, made possible by a specialized AI computer vision algorithm, provides personalized interactive feedback to children as they experiment and make discoveries in their physical environment, fostering their curiosity and 21st century skills like critical thinking, collaboration, and persistence.

Improving science, technology, engineering, and mathematics (STEM) teaching is crucial for improving STEM learning. Yet teacher training improvements progress slowly. And even the best teachers are challenged to maintain the attention of new cohorts of “digital natives” and feel the need to find innovative ways to engage them. Less focus on scientific facts and more experiences with scientific inquiry better engage the natural curiosity of children. But many elementary teachers typically do not have the background or curriculum materials to teach science from an inquiry perspective.

Addressing these challenges, we have been developing mixed-reality Intelligent Science Stations to engage children in active, inquiry-based experimentation and learning experiences in the physical world while providing interactive guidance that supports teachers as well as students. Children perform and interpret real-world experiments in a given physical apparatus (e.g., an earthquake table, ramps, a balance scale). Artificial intelligence (AI) computer vision algorithms reconstruct the physical scene and provide input to pedagogical algorithms that track the children’s progress and provide adaptive, automated feedback to guide them in scientific inquiry, producing a powerful form of active-learning support.

Along with substantial consensus around the power of active learning, comes some lack of precision in what its essential ingredients are. New educational technologies offer vehicles for systematically exploring benefits of alternative techniques for supporting active learning. We introduce a new genre of Intelligent Science Station technology that uses Artificial Intelligence (AI) to support children in learning science by doing science in the real world. We use this system in a randomized controlled trial that investigates whether active learning is best when it is implemented as guided deliberate practice, as constructive “hands-on” activity, or as a combination of both. Automated, reactive guidance is made possible by a specialized AI computer vision algorithm we developed to track what children are doing in the physical environment as they do experiments and discoveries with physical objects.

The results support deliberate practice and indicate that having some guided discovery based on effective learning mechanism such as self-explanation, contrasting cases and personalized interactive feedback produces more robust learning compared to exploratory construction alone. Children learning through guided discovery achieve greater understanding of the scientific principles than children learning through hands-on construction alone (4 times more pre-to-post test improvement). Importantly, a combined guided discovery and hands-on construction condition leads to better learning of the very hands-on construction skills that are the sole focus of the hands-on constructive learning condition (>10 times more pre-to-post improvement). These results suggest ways to achieve powerful active learning of science and engineering that go beyond the widespread temptation to equate hands-on activity with effective learning.

The possibility of leveraging technology to support children's learning in the real world is both appealing and technically challenging. We have been exploring factors in tangible games that may contribute to both learning and enjoyment with an eye toward technological feasibility and scalability. Previous research found that young children learned early physics principles better when interactively predicting and observing experimental comparisons on a physical earthquake table than when seeing a video of the same. Immersing children in the real world with computer vision-based feedback appears to evoke embodied cognition that enhances learning.

In the current experiment, we replicated this intriguing result of the mere difference between observing the real world versus a flat-screen. Further, we explored whether a simple and scalable addition of physical control (such as shaking a tablet) would yield an increase in learning and enjoyment. Our 2x2 experiment found no evidence that adding simple forms of hands-on control enhances learning, while demonstrating a large impact of physical observation. A general implication for educational game design is that affording physical observation in the real world accompanied by interactive feedback may be more important than affording simple hands-on control on a tablet.