A number of folks have responded to my post on merit pay for teachers.
Laura Chern pointed out that any sort of merit pay system relies on subjective opinions of superiors, and they can be biased and play favorites.
Larry Winkler observed that, at best, standardized tests are an imperfect measure of student achievement. They can only measure what they are designed to measure and that will inherently be some subset of what we think of as mastering academic material. There will be multiple ways of testing knowledge of the same material, and different kinds of tests will yield different results. So, it is a mistake to assume that the results of any single test will provide reliable information on the relative understanding of different students of the tested material.
I understand Larry to say that we would compound that mistake if, in addition to assuming that test results provide a robust measure of student mastery of a topic, we further assume that we can hold individual teachers responsible for students’ performance on the tests and calibrate individual teachers’ levels of pay to the students’ relative performances. Plus, Larry states that the value-added approach that has received the most attention contains a number of methodological flaws.
TJ Mertz adds to Larry’s points that there are a number of teachers whose students do not take standardized tests in their area and so a merit pay system wouldn’t work for them. TJ also makes the valid point that it’s a bit frightening to imagine MMSD School Board members in charge of putting together a merit pay system, when their (our) discussion of the limited value-added analysis the district currently undertakes does little to inspire confidence in their (our) grasp of the issues.
I had mentioned that I did not see what was wrong with teaching to the test, and commenters told me. The form of the test will affect the teaching. If teachers’ pay depended on test results, some would be tempted to just drill on the kinds of discrete facts that will turn up on the tests rather than on broader understanding. Also, as Larry Winkler pointed out, teachers could concentrate their efforts only on those students who have a realistic chance of improving their performance by moving up to the next category of proficiency, and pay relatively less – or no – attention to students who are already at the advanced level or those who are so far behind that it is unlikely that they will be able to improve their performance enough to move up a category. Another criticism of teaching to the test, pointed out to me by my son, is that teachers could spend their time working on test-taking skills rather than on content mastery.
In comments on the SIS blog, Peter Gascoyne observed that relying on test results is complicated and likely to be misleading. Instead, he proposed a system whereby 50% of a teacher’s pay increase would be handled as it is currently, uniformly distributed across all teachers according to tenure, grade, etc., and 50% of the pay increase would be determined by the school principal, to be allotted as she or he deems appropriate.
Shifting the focus of the discussion a bit, an anonymous poster on SIS rejoined that if the idea is to focus on changes intended to enhance student achievement, perhaps we should look at whether MMSD teachers are using their seniority rights with respect to transfers in order to teach in less challenging schools, while leaving less experienced, and presumably less skilled, teachers to fill the spots in the more challenging schools where students need the most help.
This point has some unanswered empirical dimensions, including whether there is a significant difference in levels of teacher experience across our MMSD schools and, if so, whether there is a correlation between schools with more experienced teachers and less challenging students, perhaps measured by a relatively low percentage of students who qualify for free and reduced lunch. Peter Gascoyne responded that he had looked at this very question five or six years ago and found that there was some but not too much correlation between levels of teaching experience and low percentages of poverty at our schools.
It is gratifying for me when something I write prompts responses from others who are more knowledgeable on the topic than me (not all that hard) and who broaden and deepen the discussion.
Despite all the commentary, I am still sympathetic to the idea of merit pay – in the abstract, how could anyone not be? I appreciate that any system for implementing a system of merit pay has to address the trade-off between subjective judgments of teacher quality by principals and others, with the inherent risk of bias, and objective measures of student achievement, with all the limitations inherent in any such measures. I also recognize the possibility that it may simply be impossible as a practical matter to devise a workable and fair merit pay system, though it is certainly way to early to throw in the towel on the effort.
An individual school district like ours is not going to have sufficient expertise to develop a merit pay system from scratch. For example, I think that the fairly technical points Larry Winkler makes about the limitations of particular tests are probably beyond the capabilities of our district to assess and resolve. I imagine that the nuts and bolts of teacher merit pay systems will have to be hashed out on bigger stages than ours, and that any such system would not be ripe for adoption in Madison until after it has been subject to some pretty thorough vetting by those knowledgeable in the field. (Enough metaphors in that last sentence for you?) This isn’t an area where we want to be pioneers.
With the movement toward Common Core national educational standards, which have been adopted in Wisconsin, I expect we’ll also see a movement toward a national assessment model for measuring student learning that is keyed to those standards. The reliability of such a test and the justification for utilizing its results in evaluating teacher performance will be debated on a national level. I expect that we’d want to wait to see how the issues shake out before even attempting to incorporate test results in our teacher evaluation. This is another reason to think that these sorts of changes are far from imminent in Madison.
I also suspect that any system of merit pay that we’d eventually consider in Madison would place less reliance on standardized test results that many critics of such systems seem to fear. My sense is that the results would be one factor of many that would be taken into account in assessing a teacher’s performance, and that test performance would only have a material impact if the results varied significantly from the norm, in either direction.
I am grateful for the comments. I think they have deepened my understanding of the issue and enhanced my appreciation for the challenges inherent in devising a sensible system of merit pay for teachers. I hope others will continue to respond to what I write.