Leslie Hawley, a postdoctoral trainee with the CYFS Statistics and Research Methodology Unit, will receive a University of Nebraska-Lincoln research award that recognizes her dissertation’s potential to improve the evaluation of K-12 teachers.
UNL’s Phi Delta Kappa Chapter recently announced Hawley as the recipient of its Dr. Ron Joekel Research Award, which honors outstanding education-focused research conducted by a UNL graduate student. Hawley, who completed her doctorate from UNL’s Qualitative, Quantitative and Psychometric Methods program in December 2012, will receive the $1,500 award at a fall meeting of the PDK Chapter.
Testing Ground
Hawley’s dissertation outlines a methodology that could more precisely measure the performance of teachers, whose job security increasingly rests on student assessments, she said.
According to Hawley, many states evaluate teachers using “value-added” performance measures that place great weight on students’ standardized test scores, which often account for up to 50 percent of a teacher’s evaluation.
Combining this information with other factors to construct a single evaluation score, however, requires making decisions and assumptions about data that can diminish the stability of the outcomes, Hawley said. Her dissertation subsequently found that different methods of merging information can lead to significant fluctuations in where teachers rank among their peers.
Given this, Hawley explored the potential of “latent variable” methods – which incorporate multiple test scores – to reduce measurement error and produce more consistent estimates of teacher effectiveness.
“No method is perfect,” said Hawley, “but I would hope that, if we had 300 teachers, it would be the difference between ranking a teacher 255th or 250th. But if a teacher is ranked 250th under one set of assumptions and ranked second under another set, that’s a problem – one that was sometimes present in the results.
“When you use latent variable methods … you are making fewer assumptions about how the information is combined. You’re essentially letting the data do a little bit more of the talking.”
Hawley said her results provided support for this hypothesis, with the latent variable methods yielding more stable teacher rankings across conditions examined in her study. This indicates that the latent variable approach could reduce the chances of undervaluing quality teachers and overrating substandard educators, she said.
Teaching Experience
Hawley noted that growing up with a mother who served as a teacher – and working as a social studies teacher herself – has provided plenty of incentive to ensure that educators are evaluated as fairly and accurately as possible.
“We’re making very high-stakes decisions. I feel very strongly that if we’re going to make these decisions – and states are going to use these methods – that they need to be done very carefully and cautiously,” said Hawley, who emphasized that student test scores are influenced by many factors beyond classroom instruction. “The methods need to be fleshed out and rigorously evaluated for all kinds of potential scenarios.”
She also credited the evolution of her dissertation to James Bovaird, who chaired her dissertation committee and directs the CYFS Statistics and Research Methodology Unit.
“I would not have gotten here without Jim’s guidance; he showed me how to think about these issues from a broader perspective,” Hawley said. “He doesn’t carve out the path for you, but he leads you in a direction and lets you explore.”