8  Letter Sound Knowledge (ROAR-Letter)

ROAR-Letter measures knowledge of uppercase and lowercase letter names and sounds. ROAR-Letter is designed to run as a computer adaptive test (CAT) to assess where a student is on the continuum of letter sound knowledge. ROAR-Letter can also be used as a diagnostic measure to guide teaching as it returns specific information about the letter names and letter-sound correspondences that the student does and does not know.

8.1 Structure of the task

ROAR-Letter is a four alternative forced choice (4AFC) task divided into 3 blocks:

• Uppercase letter names (26 items)
• Lowercase letter names (26 items)
• Letter-sound correspondences (36 items)

Like all ROAR measures, ROAR-Letter is lightly gamified. The task begins with instructions and practice trials with feedback until the student understands the game. Then, in each block, the student is presented with the name or sound of a letter and asked to select the correct letter from the four choices (see Figure 8.1).

Figure 8.1: ROAR-Letter task. In the ROAR-Letter task, participants are first directed to choose the spoken letter name and then the letter sound in separate blocks. The task is divided into several blocks with encouragement throughout. Students have the opportunity to replay any letter name or sound audio they miss. There is no time limit for responses.

8.2 Design and implementation of computer-adaptive letter-sound assessment

A computer-adaptive version of ROAR-Letter is planned for release in fall 2024 and will be more efficient and reliable with fewer items. To determine the optimal ROAR-Letter CAT we first collected data from 1986 students in kindergarten, first, and second grade. Table 18.1 shows the demographics of the participants and Table 8.1 shows the characteristics of the schools that participated. 1217 students were administered the full version with all 88 items with no time limit (26 lowercase letter-name items; 26 uppercase letter-name items; 36 letter-sound correspondences). 769 students were administered a shorter version with 10 letter-name items (5 lowercase and 5 uppercase) and up to 36 letter-sound correspondences, with a 5 minute time limit. Using student data, we ran a CAT simulation to a) determine how these students would have responded under different item-selection criteria and b) choose the number of items and optimal CAT parameters to achieve reliable estimates of letter-sound knowledge in the fewest number of trials. Based on this data we were able to develop an efficient and precise computer adaptive test of letter names and letter-sound knowledge (see Chapter 18).

	N
Median Students	427
Median Free or Reduced Lunch	247
Race/Ethnicity
Median Hispanic Ethnicity	66
Median White	103
Median Black or African American	3
Median Asian	25
Median American Indian or Alaska Native	0
Median Hawaiian or Other Pacific Islander	1
Median Multiracial	797
Organization Type
Public School	21
Charter School	3
Private School	1
Summer School/Tutor Program/Other	2

Table 8.1: Characteristics of participating schools in the calibration sample.

8.3 Scoring

ROAR-Letter is a four alternative forced choice (4AFC) task and items are scored as correct or incorrect (dichotomous scoring) by comparing the participant’s response (mouse click/touchscreen indicating the chosen letter), to the correct answer. Each response is scored in real time. The participant’s ability or Scaled Score (\(\theta\)) on ROAR-Letter is computed based on an item response theory (IRT) model. IRT puts ability (\(\theta\)) on an interval scale meaning that Scaled Scores scores can be compared over time and across grades. A CAT algorithm is used to optimize the precision and efficiency of ROAR-Letter (see Chapter 18). For more information on interpreting scores see Section 5.2 and Section 3.1.

In addition to a Scaled Score, ROAR-Letter also reports the letter names and letter-sound correspondences that a participant does and does not know. This information can be used for individualizing instruction and/or creating instruction groupings.