Rapid Online Assessment of Reading (ROAR)

Technical Manual

A bridge between the lab and the classroom

Assessments are typically time-consuming and resource-intensive to administer: Individually administering assessments to each student in a classroom means a substantial amount of lost instruction time and requires extensive training for teachers to accurately administer and score measures that are used for high-stakes decisions (e.g., access to intervention). Researchers face these same challenges creating a bottle-neck to research at scale. While education technology companies have built products that lower the demands on teachers, many of these products are expensive, grounded in opaque, proprietary technology, and lack a strong research backing. Hence, these products rarely get used in research, creating a disconnect between educational research and practice.

We launched ROAR envisioning a new model: an open-source, open-access assessment platform, grounded in ongoing academic research, and co-developed in collaboration with school-district stakeholders. Rather than a one-way street from the lab to society (often with a commercial intermediary), ROAR’s goal is to inculcate a virtuous cycle between research and practice. We aim to build a suite of completely automated, lightly gamified, online assessments that are grounded in ongoing cognitive neuroscience research and validated against the current “gold standard” of standardized, individually-administered assessments. Our approach is to partner with school districts and community based organizations at each stage of research and development to ensure that our research is grounded in real-world problems and inspired by the deep knowledge of educators who work with children and youth across a diversity of contexts. Through this “Research Practice Partnership” model, we endeavor towards a new assessment methodology that is more valid, precise, efficient, and informative. We aim to design this platform around the diversity of learners in the United States (and abroad). We prioritize transparency at every stage: whenever feasible, materials and technology are made public and each measure within ROAR is published in open-access, peer-reviewed journals with the goal of building more systemic connections between the lab, classroom, and society.

Acknowledgements

ROAR reflects the collective vision and dedication of an incredible team of collaborators. This work would not have been possible without the hundreds of teachers, reading specialists, school administrators, parents, and community based organizations that believed in the ROAR mission and collaborated at each stage of ROAR research and development. We would like to specifically thank Ching-Pei Hu and Belmont-Redwood Shores School District, John Baker, Anna Herrera and Redwood City School District, and Marta Batlle for their partnership on early validation studies. Additionally, dozens of academics have made important contributions along the way including Ben Domingue, Rebecca Silverman, Joshua Lawrence, Mike Frank, Jasmine Tran, Clementine Chou, Amy Burkhardt, Maya Brunton, Aryaman Taore, Kenny Tang, Alby Ungashe, Klint Kanopka, Yiqing Liu and others.

ROAR would not have been possible without generous funding from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD095861), Advanced Educational Research and Development Fund (AERDF), Stanford-Sequoia K-12 Research Collaborative, Microsoft, Stanford Impact Labs, Neuroscience:Translate, Klingenstein Foundation, Tools Competition. Rapid Visual Processing measures (9.2.2 Rapid Visual Processing) were developed in collaboration with the UCSF Multitudes at the Dyslexia Center with funding from the State of California.

© Jason D. Yeatman, Stanford University