Background: Published studies
Bhat et al. (2024) reported marginal reliability of 0.85 for ROAR-Fonema (see more detail in Section 20.1).
Reliability of fixed-length ROAR-Fonema
Table 22.1 reports marginal reliability computed based on data from 4129 students under the IRT model for 2 different versions of ROAR-Fonema: the short, 20 item version and the long 54 item version. The computer-adaptive ROAR-Fonema is planned for release in fall 2024 and will be more efficient and reliable with fewer items. Reliability (\(\rho_{xx^\prime}\)) is computed based on the estimated variance of \(\hat{\theta}\) relative to the estimated standard error (\(\widehat{SE}(\hat{\theta})^2\)) using Equation 22.1:
\[
\hat{\rho}_{xx^\prime} = \frac{\widehat{VAR}(\hat{\theta})}{\widehat{VAR}(\hat{\theta}) + \widehat{SE}(\hat{\theta})^2},
\tag{22.1}\]
To ensure that ROAR-Fonema is fair and equitable for different demographic groups, we also report reliability by gender (Table 22.2), eligibility for free and reduced price lunch (Table 22.3), English learner status based on state of California designations (Table 22.4), primary langauge spoken (Table 22.5), special education (Table 22.6), ethnicity (Table 22.7), and race (Table 22.8).
References
Bhat, Kruttika G., Alexa Mogan, Ana Saavedra, Mia Fuentes-Jimenez, Julian M. Siebert, Wanjing Anya Ma, Carrie Townley-Flores, et al. 2024.
“Shared and Unique Influences of Phonological Processing on Reading and Math.” OSF Preprints. https://doi.org/
https://doi.org/10.31219/osf.io/em3bg.