Using Randomized Experiments as a Methodological and Conceptual Tool for Improving the Design of Online Learning Environments

Abstract

Although there is often a convoluted pipeline from educational research to practice, a commonly touted benefit of MOOCs and online learning is the opportunity to bridge this gap. Drawing on theory and methodology from cognitive science and education research, this paper proposes that experimental comparisons – which evaluate how target learning outcomes are influenced by changing the components of an online resource – can play a special role in bringing together research and practice. Three examples of experiments in online environments (MOOCs) are presented and explained, spelling out the logic and considerations in embedding experiments in MOOCs. We also consider how and why the conceptual tool of experiment-focused design – considering hypothetical experiments during the process of designing online instruction – is valuable from a practical perspective, because it can guide decisions about instructional design, provides a basis for evidence-based improvements to students’ outcomes, and supports iterative improvement of online educational resources.