Background: Published studies
Bhat et al. (2024) reported marginal reliability of 0.85 for ROAR-Fonema (see more detail in Section 24.1).
Reliability of fixed-length ROAR-Fonema
Table 26.1 reports marginal reliability computed based on data from 4128 students under the IRT model for 2 different versions of ROAR-Fonema: the short, 20 item version and the long 54 item version. The computer-adaptive ROAR-Fonema is planned for release in fall 2024 and will be more efficient and reliable with fewer items. Reliability (\(\rho_{xx^\prime}\)) is computed based on the estimated variance of \(\hat{\theta}\) relative to the estimated standard error (\(\widehat{SE}(\hat{\theta})^2\)) using Equation 26.1:
\[
\hat{\rho}_{xx^\prime} = \frac{\widehat{VAR}(\hat{\theta})}{\widehat{VAR}(\hat{\theta}) + \widehat{SE}(\hat{\theta})^2},
\tag{26.1}\]
To ensure that ROAR-Fonema is fair and equitable for different demographic groups, we also report reliability by gender (Table 26.2), eligibility for free and reduced price lunch (Table 26.3), English learner status based on state of California designations (Table 26.4), primary langauge spoken (Table 26.5), special education (Table 26.6), ethnicity (Table 26.7), and race (Table 26.8).
References
Bhat, Kruttika G., Alexa Mogan, Ana Saavedra, Mia Fuentes-Jimenez, Julian M. Siebert, Wanjing Anya Ma, Carrie Townley-Flores, et al. 2024.
“Shared and Unique Influences of Phonological Processing on Reading and Math.” OSF Preprints.
https://doi.org/10.31219/osf.io/em3bg.