1  ROAR Vision and Mission

ROAR emerged out of more than a decade of research in the Brain Development & Education Lab. Through a partnership with the Language to Literacy Lab, ROAR extended this work to develop additional assessments that leverage the extensive literature of cognitive, psycholinguistic, and developmental research into the language components of reading comprehension. Our goal was to develop a completely automated, lightly gamified online assessment platform that could assess an entire school district in the time typically required to administer an assessment to a single student. We envisioned a new approach to research and development, grounded in the principles of open-science, where each ROAR measure would be grounded in the extensive interdisciplinary literature on reading development, be validated adhering to the highest standards of rigor in each discipline, and be published in open-access journals to support scientific transparency.

1.1 Open-source ideology in educational assessment

The last decade has seen a revolution in scientific transparency. The open-science movement began as a grassroots movement to make science more transparent, accessible and reproducible through the open sharing of code and data to accompany publications in open-access journals. The success of the open-science movement can be appreciated in new public mandates for data sharing by many of the major scientific funders in the United States and Europe, as well as proliferation of organizations like the Center for Open Science, and preprint servers like bioRxiv, that all make it easier to document, share and reproduce scientific research. In fields like cognitive neuroscience, it is now standard practice for software and algorithms to be open-source, and many journals even require various open-science practices. However, in education, most widely used assessments are grounded in proprietary products, with many of the technical details guarded by paywalls or made purposefully opaque to maintain a competitive edge in the market. There are, of course, counterexamples like DIBELS that have always maintained open-access printed materials, and with projects like the Item Response Warehouse which provides open access to many educational datasets, there is a clear desire among many educational researchers for a move toward open science.

We launched ROAR with the mission to bring the open-source ideology to educational assessment. Our lab has a long track record of developing and supporting open-source software for analysis and sharing of brain imaging data, and for modeling the interplay between brain development and learning. ROAR represents the next phase of this open-science mission: to build tools that fill the needs of educators to assess reading development while, simultaneously, opening the door to research at an unprecedented scale. Not every aspect of ROAR is completely open, but we consciously prioritize open science at every stage of development including this technical manual which is written as an open-source quarto book.

1.2 The ROAR Assessment Suites

ROAR consists of a collection of measures, each designed to tap into a critical aspect of reading. Each individual measure can be run independently and returns raw scores, standard scores, and percentiles relative to national norms. Additionally, measures are also grouped into measurement suites that comprehensively evaluate different constructs in reading development, and produce composite scores and risk indices.

1.2.1 ROAR Foundational Reading Skills Suite

The ROAR Foundational Reading Skills Suite provides efficient screening of the core decoding and word recognition skills that underlie reading development, including phonemic awareness, letter knowledge, word recognition, and reading fluency. These measures are designed to support early identification of students at risk for reading difficulties, including dyslexia.

1.2.2 ROAR Comprehension Suite

The ROAR Comprehension Suite provides efficient, automated assessments of the language comprehension component of reading ability — the skills needed to make meaning from text once words are recognized. The suite includes assessments of vocabulary, morphological knowledge, syntactic knowledge, and inferential reasoning. Results from the Comprehension Suite can help identify students who may benefit from additional instructional support in vocabulary, morphology, syntax, or inferential reasoning — areas that are critical for reading comprehension but may not be captured by foundational skills screening alone.

Together, the two suites allow educators and researchers to examine both the word recognition and language comprehension pathways described in Scarborough’s Reading Rope and related theoretical frameworks, providing a comprehensive picture of students’ reading development.