When educators feared learning loss heading into the 2020-21 school year, interim and formative end-of-year assessments were met with a potpourri of reactions. Depending on who you asked, assessments were critical to understanding COVID slide or unnecessary sources of stress for students whose learning losses were predictable by ZIP code, or answers landed somewhere in between — with tests being helpful, but only as part of a much larger picture that considers trauma, access and other conditions of learning.
With assessments completed and data gathered in many places, districts and testing experts are working through some challenges and potential solutions for the next round.
"There have been lots of challenges, not very many solutions," said Steve Joel, superintendent of Lincoln Public Schools in Nebraska.
Remote conditions produce inaccurate data
With many districts in hybrid or remote learning modes, having students take assessments from home has been far from ideal, in some cases leading to inflated or deflated scores.
"Scores of tests taken at home tend to be higher," said Bryan Hancock, a partner at McKinsey & Company, a consulting firm working in the education sector, during a webinar hosted by the National Press Foundation. "Researchers attribute it to well-meaning parents or other factors that are hard to control during testing at home."
Other students are scoring worse than usual, which has been a point of contention with some parents, said Joel.
"What if there is a distraction at home, whether that’s another sibling?" said Anne Hyslop, assistant director of policy development and government relations at the Alliance for Excellent Education and a former senior policy advisor in the U.S. Department of Education's Office of Planning, Evaluation, and Policy Development under the Obama administration. "We also know that children have gone through additional trauma this year related to the pandemic. Are kids in the right head space that day?"
Brooke Morgan, coordinator of Innovative Learning for Alabama's Talladega County Schools, said the district's principals are seeing many students performing worse than they are capable of. "We’ve had to work hard to incentivize them and motivate them to get the best information possible," she said, adding better performance on future diagnostics could be incentivized by things like pizza or ice cream parties.
"We need our parents to supervise their child in the home environment," Joel said, adding that isn't happening to the degree expected when parents signed up for remote learning. "What we’re trying to emphasize with parents is that this is a partnership."
Parents should be informed about their role and responsibilities while their children are taking assessments remotely, Jadi Miller, director of assessment for Nebraska's Elkhorn Public Schools, told K-12 Dive in November.
Meanwhile, some districts are pushing for in-person testing. Fairfax County Schools, for example, is allowing students back in the building for assessments, giving parents the option to refuse their children's participation.
Disappearing students contributes to incomplete results
Districts might also need to collect more data related to participation rates, access, types of instruction and social-emotional well-being, said Hyslop. That data can be used to reach students who are missing.
"I think the biggest thing that many districts are facing right now, especially those that are either in hybrid or all-virtual mode at the moment, is simply the fact that a lot of students aren’t participating or engaging in school," said Hyslop. "Or, in the worst case, may be missing entirely."
Nationally, results from the NWEA MAP Growth assessment show about one in four students who had previously been tested didn't show up this year.
"We can't say for certain why they're not showing up," said Megan Kuhfeld, NWEA's senior research scientist, during a press webinar. "But this is a lot of students and it’s a very concerning finding, both in terms of being able to find these students as well as in terms of estimating learning loss or impacts."
Reaching the missing children, diagnosing their learning losses, and getting supports to those students are often challenges. RAND Senior Policy Researcher Julia Kaufman warned students who need the most support often don't take advantage of academic programs.
Hyslop suggested rigorous district outreach is needed. Sometimes, she said, sending an email or making a phone call is not enough, and district personnel have to physically reach out to students instead.
"This takes ... building trust. It takes capacity and time and money," she said. "The more I think that this can be coming with folks that already have connections with the community, the more effective it can be."
Talladega County Schools is anticipating some of its missing students will return in the spring. "We will certainly look at them on an individual basis and serve them through summer work to help them catch up," said the district's superintendent, Suzanne Lacey.
Rethinking assessment is 'messy'
Some districts nationwide are pushing to move away from assessments and their high stakes, which poses its own challenges.
"What we don’t want to do is give some standardized assessment that we know was oppressive or wasn’t culturally responsive anyway," said Luvelle Brown, superintendent of Ithaca City School District in New York. "I think it’s rocky, it's challenging, it's messy and it's awkward because we’re trying to change it. ... But it’s going to take time."
Brown's district is working to develop guidelines in the meantime and bringing in community partners and students to share their perspectives on the inequitable impact of previous assessments.
"That’s the ongoing dialogue," Brown said. "The default place for our industry, our schools, is to funnel kids into remedial programs."
Districts are also having to contend with stakes often tied to assessments, such as advanced course and gifted and talented program placement. "Those kinds of policies may need to be changed this year," Hyslop said. "We know that students of color and low-income students have been especially effected."
She suggested tying an additional data point other than assessments to those stakes, or revisiting high stakes at a later time.
Testing experts have recommended changing approaches to high-stakes assessments and detangling them from things like teacher pay raises and promotions. Instead, Chris Minnich, CEO of NWEA, said assessments' use should be narrowed to gathering student data, personalizing instruction and monitoring academic progress.
The U.S. Department of Education encouraged states in September to "rethink" assessment this year, calling now "the perfect time" for such changes. Nebraska is among states rethinking its assessment model, putting in place an interim assessment system that eliminates an end-of-year summative exam.