1  ROAR Comprehension Suite

The ROAR Comprehension Suite was developed in response to requests from school partners using the ROAR Foundational Reading Suite who sought a more holistic understanding of student reading profiles. To complement assessments of phonemic awareness, decoding, and fluency, partners needed efficient measures focused on language and comprehension skills. Grounded in research conducted in the Language to Literacy Lab, led by Dr. Rebecca D. Silverman, and research by leading language and comprehension scientists in the field, the ROAR Comprehension Suite provides critical assessments of language and reading comprehension that support educators and researchers in understanding the comprehension-related strengths and needs of students.

1.1 Theoretical Foundation

Reading comprehension occurs at multiple levels, reflecting the complex interaction between word recognition processes and higher-order meaning construction (McNamara and Magliano 2009). Because of this complexity, some researchers have suggested that reading comprehension is too multifaceted to be captured by any single theory, necessitating the use of multiple theoretical frameworks to describe its interrelated components (Perfetti & Stafura, 2017). These frameworks range from simple and parsimonious to increasingly detailed accounts of these interactions, each enhancing our understanding of reading as an integrated system of multiple component skills working together. The Simple View of Reading established the premise that reading comprehension depends on both decoding and linguistic comprehension (Gough and Tunmer 1986; Wesley A. Hoover and Gough 1990). Building on this foundation, three models provide the primary theoretical framework for the ROAR Comprehension Suite. Scarborough’s Reading Rope (Scarborough, Neuman, and Dickinson 2001) distinguishes subcomponents of decoding and linguistic comprehension, informing our organization of assessments into discrete skill areas. The Direct and Indirect Effects Model of Reading (DIER) (Kim 2020) maps how specific language and cognitive skills affect reading comprehension both directly, by contributing to meaning construction, and indirectly, by supporting word recognition processes. Share’s Combinatorial Model (Share 2025) provides insight into developmental progression, guiding our understanding of how readers advance from processing sub-morphemic units to morpho-lexical and eventually supra-lexical comprehension. Additional theoretical perspectives that inform specific assessments are discussed in their respective sections. Across these different models, it is clear that reading comprehension includes facility with specific knowledge and skills (e.g., vocabulary knowledge, morphological knowledge, and syntactic skills) as well as the ability to integrate multiple sources of information to make inferences and generate deep understanding of text (Blum et al. 2020; Cain and Oakhill 1999; Kendeou 2015; Trabasso 1980). The ROAR Comprehension Suite is grounded in these theories of reading comprehension as well as the empirical research that supports them. These theoretical frameworks have been empirically validated in studies across languages and language backgrounds, including multilingual learners.

1.2 Language and Reading Comprehension Components

The ROAR Comprehension Suite is intended to assess individual components of language and reading comprehension and provide researchers and schools with information to guide instruction. In addition to foundational reading skills needed for decoding, language comprehension skills are needed for students to make meaning of the texts they encounter (Gough and Tunmer 1986; W. A. Hoover and Gough 1990; Scarborough, Neuman, and Dickinson 2001). Language comprehension is the ability to understand spoken or written language, including how word parts affect meaning (morphology), what words mean across contexts (vocabulary), how words are combined to form sentences (syntax), and how to draw conclusions from text by connecting ideas and deducing meaning (inference). Together, decoding and language comprehension are core components of reading comprehension, the ultimate goal of reading (Duke, Ward, and Pearson 2021; W. A. Hoover and Gough 1990). Unlike other comprehension assessments that feature long reading passages, the Comprehension Suite uses brief passages and targeted sentences to obtain valid and reliable measures of these subskills while maximizing assessment efficiency. 

1.3 Assessment Development Framework

To bridge theoretical understanding with practical assessment needs, we employed the BEAR Assessment System framework (BAS) (Wilson 2005, 2023), a comprehensive approach to educational measurement developed at the Berkeley Evaluation and Assessment Research Center. This framework was particularly valuable for developing the ROAR Comprehension Suite because it structured our work through four interconnected components: construct mapping, item design, outcome space development, and model selection. This systematic approach ensures that ROAR Comprehension measures provide educators and researchers with information about a student’s location on a developmental pathway for each assessment.

1.4 Suite Organization: Oral and Written Comprehension

The ROAR Comprehension Suite assesses comprehension through two modalities: oral comprehension and written comprehension.

1.4.1 Oral Language Comprehension

• Syntax: Evaluates understanding of English sentence structures through a listening (receptive) and picture-matching task

1.4.2 Written Language Comprehension

• Written Vocabulary: Measures understanding of word meanings in written contexts

• Morphology: Assesses the ability to recognize and manipulate meaningful word parts

• Inferencing: Evaluates the ability to draw conclusions from information presented in written text and understand relationships between ideas that may not be explicitly stated

This organization allows educators to distinguish between students’ oral comprehension skills, which do not require reading, and their written comprehension skills, which involve processing printed text. This distinction is particularly valuable for identifying whether students’ comprehension difficulties stem from general language processing challenges or are specific to reading contexts.

1.5 Assessment Specifications

Table 1.1 lists assessments in the ROAR Comprehension Suite that are currently undergoing pilot testing in our partner schools:

Table 1.1: Overview of assessments in the ROAR Comprehension Suite

Assessment	Purpose	Grades	Format	Multiple Choice Task	Time
Syntax	Assess understanding of English sentence syntax, including phrases and clauses	K-12	listening	Listen to phrases and sentences of varying complexity and choose the matching picture from an array.	5 min
Morphology	Assess understanding and ability to use words with prefixes and suffixes	2-12	sentence reading	Choose the best word to complete a fill-in-the-blank sentence.  The choices have the same base and different prefixes or suffixes.	5 min
Written Vocabulary	Assess understanding of written vocabulary in context	2-12	sentence reading	Read a sentence with one word in bold. Choose the word whose meaning is closest to the bold word.	5 min
Inference	Assess ability to understand the literal meaning of short passages and to use inferences to make connections that are not explicitly stated.	2-5	short passage reading	Read a short passage and select the answer to a literal or inferential question.	10 min

We are currently working to create a computer-adaptive version of each measure, which may change the timing for each assessment.

1.6 Standards Alignment

The ROAR Comprehension Suite was designed with attention to national standards frameworks, ensuring that assessment results provide meaningful, actionable information within the accountability contexts in which educators operate. Each assessment maps onto Common Core State Standards (CCSS) anchor standards for English Language Arts and the NAEP Reading Framework’s cognitive targets (National Assessment Governing Board 2021), reflecting the constructs prioritized across these frameworks as foundational to literacy development. Specific alignment is detailed in each assessment’s technical section; Table Table 1.2 provides an overview across the suite.

Alignment of ROAR Comprehension Suite Assessments with CCSS and NAEP Frameworks

Table 1.2: Note. NAEP cognitive targets are drawn from the NAEP Reading Framework (National Assessment Governing Board 2021), which identifies three targets: Locate and Recall, Integrate and Interpret, and Critique and Evaluate.

Assessment	Key Construct	CCSS Anchor Standards	NAEP Cognitive Target
Syntax	Syntactic comprehension	CCSS.ELA-LITERACY.CCRA.L.1, L.3	Integrate and Interpret
Written Vocabulary	Vocabulary in context	CCSS.ELA-LITERACY.CCRA.L.4, L.6	Locate and Recall; Integrate and Interpret
Morphology	Morphological knowledge	CCSS.ELA-LITERACY.CCRA.L.4	Locate and Recall
Inference	Inferential reasoning	CCSS.ELA-LITERACY.CCRA.R.1, R.2, R.3	Integrate and Interpret

1.7 Developmental Appropriateness and Grade-Level Recommendations

The ROAR Comprehension Suite is designed to supplement the ROAR Foundational Reading Suite by providing additional information about students’ abilities in the areas of vocabulary, syntax, morphology, and inference. The assessments are intended for use as universal screening tools to identify students who may benefit from additional instructional support in the targeted areas or consideration for further in-depth testing.

1.8 Integration with ROAR Foundational Reading Suite

The ROAR Comprehension Suite is specifically designed to complement the ROAR Foundational Reading Suite (ROAR-Phoneme, ROAR-Letter, ROAR-Word, and ROAR-Sentence) by providing insights into the language comprehension component of reading ability. While foundational skills focus primarily on the word recognition pathway, comprehension skills address the linguistic comprehension pathway, since both are essential for reading success.

This complementary relationship allows educators to:

• Detect comprehension difficulties: Distinguish between word recognition challenges and language comprehension challenges

• Plan targeted instruction: Identify specific areas within language comprehension that need support

• Monitor development across skills: Ensure students are developing both foundational skills and comprehension abilities

• Support diverse learners: Recognize that some students may have strong foundational skills but need support in language comprehension, or vice versa

1.9 Limitations and Considerations

The ROAR Comprehension Suite assessments do not diagnose reading disability and should not be used as part of a psycho-educational evaluation or to qualify students for special education. At this time, these assessments are only available in English, though development of Spanish versions is under consideration.

The assessments are designed as universal screening tools and should be interpreted alongside other sources of evidence, including classroom observations, curriculum-based measures, and comprehensive diagnostic assessments when indicated. Professional development for educators using these measures should emphasize appropriate interpretation and instructional applications.

1.10 Future Developments

Ongoing research and development efforts for the ROAR Comprehension Suite include:

• Implementation of computer-adaptive testing to improve efficiency and precision

• Expansion of the inference assessment to higher grade levels

• Addition of new comprehension components based on emerging research

References

Blum, Alexander Mario, James M Mason, Jinho Kim, and P David Pearson. 2020. “Modeling Question-Answer Relations: The Development of the Integrative Inferential Reasoning Comic Assessment.” Reading and Writing: An Interdisciplinary Journal 33 (8): 1971–2000.

Cain, Kate, and Jane V Oakhill. 1999. “Inference Making Ability and Its Relation to Comprehension Failure in Young Children.” Read. Writ. 11 (5): 489–503.

Duke, Nell K, Alessandra E Ward, and P David Pearson. 2021. “The Science of Reading Comprehension Instruction.” The Reading Teacher 74 (6): 663–72.

Gough, Philip B, and William E Tunmer. 1986. “Decoding, Reading, and Reading Disability.” Remedial and Special Education 7 (1): 6–10.

Hoover, W. A., and P. B. Gough. 1990. “The Simple View of Reading.” Reading and Writing 2 (2): 127–60. https://doi.org/10.1007/BF00401799.

Hoover, Wesley A, and Philip B Gough. 1990. “The Simple View of Reading.” Read. Writ. 2 (2): 127–60.

Kendeou, Panayiota. 2015. “A General Inference Skill.” In Inferences During Reading, edited by Edward J O’Brien, Anne E Cook, and Robert F Lorch Jr, 160–81. Cambridge, England: Cambridge University Press.

Kim, Young-Suk Grace. 2020. “Hierarchical and Dynamic Relations of Language and Cognitive Skills to Reading Comprehension: Testing the Direct and Indirect Effects Model of Reading (DIER).” Journal of Educational Psychology 112 (4): 667–84. https://doi.org/10.1037/edu0000407.

McNamara, Danielle S, and Joe Magliano. 2009. “Toward a Comprehensive Model of Comprehension.” Psychology of Learning and Motivation 51: 297–384.

National Assessment Governing Board. 2021. “Reading Framework for the 2026 National Assessment of Educational Progress.” https://www.nagb.gov/content/dam/nagb/en/documents/publications/frameworks/reading/2026-reading-framework/naep-2026-reading-framework.pdf.

Scarborough, Hollis S, SB Neuman, and DK Dickinson. 2001. “Connecting Early Language and Literacy to Later Reading (Dis) Abilities: Evidence, Theory, and Practice.” Handbook of Early Literacy Research 1: 97–110.

Share, David L. 2025. “Blueprint for a Universal Theory of Learning to Read: The Combinatorial Model.” Reading Research Quarterly 60 (2): e603.

Trabasso. 1980. “On the Making of Inferences During Reading and Their Assessment.”

Wilson, Mark. 2005. Constructing Measures: An Item Response Modeling Approach. Mahwah, NJ: Lawrence Erlbaum.

———. 2023. “Constructing Measures,” May. https://doi.org/10.4324/9781003286929.