This blog post is actually an extended comment to Elizabeth Yang's earlier Grades and Education post, which is actually a response to the NY Times' article Mayor Says Student Scores Will Factor Into Teacher Tenure.
Elizabeth posts a couple challenging questions in her post, mainly: (1) "Can you really judge how "good" a teacher is by looking at their students' standardized test scores?" and (2) "Are grades the best measure of what a teacher can do?" She suggests that yes, you can judge a teacher is by looking at test scores... but that improvements in scores should be looked at, rather than absolute scores. The second question is more rhetorical, but she mentions that quantitative measures are really what matters today.
My take on the issue is mixed. I don't have super great numbers, and I know great people who don't have awesome numbers. I also know really good teachers who take on below average classes and consequently don't have the best class grades either.
From my limited pedagogical experience, and my experience as a student, I do think that elementary and secondary school standardized test scores do reflect how good a teacher, school, or district is. But it only does so to a limited degree, because test scores aren't only correlated to teacher performance but also to confounding factors, such as socioeconomic status, availability of afterschool programs and environmental safety. Thus, I think that proper variability analysis and testing design should be undertaken before interpreting the scores. For instance, it may be possible to better judge how well a teacher is performing by comparing teachers within a certain school or geographic area.
I don't think that looking at improvement necessarily demonstrates teacher performance either. On one hand, some states have shown "improvement" by actually decreasing their standards, according to a study by the National Center for Educational Statistics. On the other hand, others have gotten "worse" because they set their educational bar higher. However, an even greater problem with using improvement to demonstrate performance is actually with teachers cheating on standarized tests... through setting "nonstandard [testing] practices and conditions" and lax oversight by schools wanting more money. Because of this, I would actually advocate using the absolute scores to evaluate student and teacher performance, again adjusting for noneducational factors, as mentioned before.
With all of that said, I don't think scores are the only way to judge performance; non-numerical factors should also play a role if jobs are at stake. For one, parental input should be considered. If parents think that a teacher is doing great, then that should be considered alongside that teacher's classes' test scores. (On a side note, I thought it was pretty ridiculous that parents had an uphill battle to fundraise to hire teaching aides in New York.) For another, I think even students should be given a role in speaking for teachers, even despite the possibility of classroom duress.
Elizabeth is right in that, realistically, only numbers really matter. When all of us write grant proposals, the funder will always ask us how we intend to measure the impact of our proposed program. The expected answer is almost ALWAYS numerical, and often based on pre- and post-test or survey scores. Though there are a lot of difficult-to-measure quantities, such as level of awareness, and controversy on where to set frames of reference, there's really no way for evaluators to easily discern an overall story from large datasets without looking at just numbers.
... Thankfully admissions committees go through each case individually.
