With its first successful online field test in 450 schools this past spring, the Nation's Report Card is one step closer to remote proctoring and administration.
The long-term goal for remote test administration was fast-tracked by the National Assessment Governing Board and the National Center for Education Statistics, which oversee and administer the National Assessment of Educational Progress, after the COVID-19 pandemic made it infeasible to give the test in 2021.
“The pandemic and NAEP’s inability to administer the 2021 assessment provided sobering lessons,” wrote Peggy Carr, commissioner of the National Center for Education Statistics, and Lesley Muldoon, executive director of NAGB, in a March 2022 blog update.
In June 2022, NCES for the first time provided a timeline of its innovation plans, which include NAEP shifting to online administration in 2024, remote proctoring and administration conceivable sometime after 2026, and adaptive testing in 2028.
The test will still be ready for online administration in 2024, said Dan McGrath, acting associate commissioner for the National Center for Education Statistics' Assessment Division, in an update to K-12 Dive on Aug. 17.
"The field test, I think, was actually quite successful," McGrath said. "And it got us in good position for online administration in 2024."
Field test will lead to changes in communication, field staff
The spring 2023 field test provided the 450 randomly selected schools with four variations of administration, meant to account for schools with different testing capacities. "It was really a test of internet access," said McGrath. "The vast majority of schools [tested] already had the capability to support this."
In the field test, 82% of schools passed screenings that indicated they had the internet capability to administer NAEP on their networks. Out of those schools, nearly three-quarters carried out a successful online assessment.
However, the test also highlighted changes NCES will have to make as it prepares for online administration in 2024.
"What we found was that we could improve our communications and preparations with schools to make sure that we had the best understanding of their capability going into their assessment," said McGrath. For example, in one school without an elevator, it was difficult to move testing equipment between floors, a challenge that could have been worked around with better communication prior to the test.
And while NCES had originally planned to pull out a majority of its field staff by 2026, the field test showed that may not be possible without overly burdening schools to provide their own staff for scenarios like supervising separate testing accommodations.
"As a result, we’re still going to reduce the number of field staff we're sending to schools," said McGrath. "But not by as many as we had initially planned."
In general, NCES now sends around three field staffers per school. In the future, it will be looking to send around two — or more when needed. That would reduce field staff by about a third or a quarter, McGrath said. Originally, it had planned to cut field staff by half.
“Our team has learned valuable lessons from the recent field test with online administration of NAEP that will inform our work moving forward," said Carr in an emailed statement to K-12 Dive. "For example, we heard loud and clear from participants that school personnel should not face extra burdens along the way. We will honor that in 2024."
Here is an updated timeline for NCES' innovations for the Nation's Report Card, according to NCES:
-
2022
Five states participate in a pilot, testing over 1,000 in 25 volunteer schools for online administration.
-
2023
The department field tests 450 randomly selected schools meant to account for schools with different online and offline testing capacities. In the field test, 82% of participating schools passed screenings indicating they can administer NAEP online.
-
2024
NAEP expected to be ready for online administration. NCES will study the use of school's testing devices to administer NAEP, conducting a pilot for adaptive testing, and testing the use of AI to score reading.
-
2025
NAEP will field test the use of schools' testing devices to administer NAEP in more than 500 schools.
-
2026
NAEP will be device-agnostic, or able to be administered regardless of the type of device used. Schools will begin using school-based equipment for testing when they are able. NAEP will also conduct a field test for adaptive testing.
-
2028
Adaptive testing, or modifying the assessment based on students' abilities, will begin.
-
TBD
NCES will have the potential to be proctored and administered remotely.
Equity, other considerations in innovation on horizon
Some upcoming innovations come with equity considerations. NCES' use of AI in scoring, for example, is being evaluated for racial, ethnic and gender bias.
Once AI is put in place, it's possible NAEP will be scored and its results released quicker than the current six months that hand-scoring requires.
And because some schools may not be able to meet the minimum standards set for devices' testing capabilities, NCES will be prepared to continually provide its own equipment, said McGrath.
“As NCES continues to innovate, we will partner closely with educators, policymakers, and others to ensure the appropriate balance of cost, benefit, and burden, while keeping the quality and integrity of data as our top priority," said Carr.
And while remote proctoring and administration is still "on the agenda," said McGrath, "it's very, very early" in the timeline for that. Last year, Carr said that would be conceivable sometime after 2026.
"We certainly want to be capable of administering NAEP remotely if we were in another situation in which it's important for the country to know about learning and achievement but kids are not able to be at schools," said McGrath. "We really would like to be able to contribute something there. But we're really in the beginning of thinking about how to get that done."