Economists to teachers: We’ve dropped the “Deselection” and moved straight to “Fire ’em”

I had a few thoughts about the big teacher quality and VAM study that came out today that I wanted to share before they float away.

My thoughts are less about the methods of the study itself and more about how it moves so quickly from the econometrics to the policy, and how the journalist presents it in the New York Times. Bruce Baker (@schlfinance101) says that there is a lot of interesting data here, and I look forward to reading his take on it, but I didn’t feel this article fairly presented the context and limitations of any study of this sort.

I haven’t had time to digest the full study, although I do want to read it. But let me start with something that may be a bit surprising to some of my readers. I don’t have a problem acknowledging the following:

1) Good teaching helps students learn, poor teaching does not help, and can even hinder student learning
2) Teaching is to some extent a stable trait, at least it seems so in our current school environment
3) Some teachers are better than other teachers
4) Some of this variation in teaching “ability” is can be reflected in student test scores, and in long term student and adult outcomes

Teachers matter. I wouldn’t be a teacher if I didn’t feel this to be true. As a college teacher, on one hand I (every day) confront evidence of my powerlessness in the face of student circumstances. On the other, I have had a few students tell me how I have changed their lives, and I (sort of) believe them. I certainly think that some of my teachers changed my life.

I am a teacher and I try to improve. But I know that there are some people who start out better than I did, improve faster, and will always be superior. Jeannette Norden is likely a better teacher than I will ever be. But despite that truth, I don’t find competitive teacher rankings by score (such as VAM) motivating, or acknowledge that they are the best way to measure my success.

I am adamant that this approach, as reflected in the quotes in the article, is toxic and corrosive to good teaching and to improving education in our country.

Rant about the NYT article

First: “That test scores help you get more education, and that more education has an earnings effect — that makes sense to a lot of people,” from Robert Meyer, not an author, but another teacher quality researcher.

When scientists talk to the media, they often face a choice between scientific precision and being clear to the layman. It is difficult to convey the uncertainty, precision, context, and humility of a scientific finding to a lay audience, which is why many scientists don’t even try. In explaining my dissertation research, I could say “Hills look steeper when you are sad.” Or I could present it in any of the following ways:
Sad people overestimate hill slant.
Sad people overestimate hill slant relative to happy people (who also overestimate hill slant).
People who listened to sad music, or wrote a sad personal story, reported the hill slant to be steeper than people who listened to happy music, or wrote a happy story.
People who listened to music that had made them sad in past studies, and made them say that they were sad in a post-experiment questionnaire (even though they didn’t realize that the point of listening to Mahler’s Der Kindertotenleider was to make them sad) estimated the slant of an 8 degree hill to be on average 23 degrees (with a standard error of around 1.4 degrees) using a verbal measure of slant as well as a visual matching device…
What? Where are you going? Oh, that won’t fit in the 20 words you have for describing my study? Ok. How about “hills look steeper when you are sad”?

But, back to this quote. I balked at this quote, because I thought, test scores help you get more education? Really? This is treating the measurement (the test score) as the causal agent, instead of the thing the test is measuring. I did well on the SAT (and the CTBS for that matter) but did those test scores help me get more education? No. They reflected the circumstances of my life: educated, curious conscientious parents, good diet, exercise, lots of reading. When I took tests, really what those tests measured were a bit of my ability, and a lot of my own education (whether it came from books, my dad’s clipped New York Review of Books articles, or him answering a question with, “Well, do you want the five minute answer, the ten minute answer, or the hour?”). But if you take away the “test scores” and add the “education” that those test scores reflect in that sentence, it becomes both more accurate, and seemingly nonsensical (although profoundly true) “That education helps you get more education, and more education has an earnings effect.”

Which get to my big beef with this study. Just because variation in one factor (teacher VAM) helps predict variation in another (student earnings, teen pregnancy, etc) does not mean that these factors are as mutable as numbers in a model. I often roll my eyes at the “correlation is not causation” criticism, because not all correlations are equal. For example, controlling for other possible correlations (third variables that could explain mutual correlation) helps a lot in making a greater case for a causal role. But when authors of these studies jump from “relatively better correlation evidence” to “we should be firing teachers tomorrow,” I cry foul. Relatively greater education may have led to greater relative earnings in the past, but does this mean that handing out Ph.D.’s will make us all rich? No. Does it mean that sending everyone to college will make us all rich? No. Somehow these economists recognize that simply giving every child in the state some books (so they have more books in the home!) will not necessarily raise their test scores (thereby lifting them out of poverty!) but they still with a straight face proclaim that a bad teacher costs $2.5 million in lost income and should be fired immediately. A CEO makes his company run more efficiently and lowers prices through outsourcing to lower labor costs (saving his shareholders $2.5 million in payroll costs), and he is a hero. Is anyone else dizzy?

Next, the economists who did this study seem to be willfully ignorant of human psychology.

This section was chilling:

“The message is to fire people sooner rather than later,” Professor Friedman said.
Professor Chetty acknowledged, “Of course there are going to be mistakes — teachers who get fired who do not deserve to get fired.” But he said that using value-added scores would lead to fewer mistakes, not more.

Neither of these economists seem to have any idea of three consequences of these policy recommendations.

First, “Firing people sooner rather than later” could change the labor pool of people ready to replace these people who got “fired sooner.” Are people really rushing to fields that promise to fire those who don’t perform in the first year or two? Could I imagine saying this to my kids’ kindergarten teacher? Could you? Show me some results! Read, kid! Reeeeeaaaad! Maybe Chris Rock is right, we just need more fear in schools. Just tell every teacher that their job depends on every student improving every year . . . What could go wrong?

Second, for those who do enter the profession despite the increased firings, this will drastically change how they teach. They will teach more to the test. You had better hope that your tests are great, and they cover the right material, because teachers will do whatever they can to improve their test scores. Or, you could talk to many of the test makers, who tell you their tests aren’t supposed to do that.

Third, all data points are not equal, nor are they as independent as these economists think. One can use statistical techniques to compare apples to apples, and tweak dimensions to get data to seem independent, but people are social critters, and they notice what happens to other people. ” ‘Of course there are going to be mistakes’ but VAM will lead to fewer mistakes, not more,” said Professor Chetty. Not more than what? Presumably what Chetty means is more mistakes than the current approach of just leaving all these awful union-protected teachers in the classroom until they leave voluntarily. But these mistakes are not equal. Firing a beloved teacher whose VAM dips below some threshhold can send a chill through a school in a way that generic benign neglect does not. My impression of the urban schools I was in (and those that my wife and father have taught in) was that being creative and inspirational must serve as its own reward. My creative and inspirational teachers dealt with their hard work being ignored by most (but savored the imagined long term effects on their students). But when instead of being ignored, you are fired? This changes the dynamic. It wreaks havoc on any dynamic of trust in the school. And it erodes freedom, curiosity, and creativity. Reformers may say, well, “Teachers have had their freedom, their curiosity and creativity, and look what they’ve done: all these bad teachers have cost our students x trillion dollars.” Even if that is true, are more of miraculous good teachers actually willing to sign up for an atmosphere where there is more firing? Less freedom? Less room to grow? Less support for improvement? Where are the studies that ask teachers why they go into teaching? Why do they stay? I doubt it’s the money.

The tone of the final few paragraphs really got me. The reporter gave Jesse Rothstein a quote, as he made my point 3 above:

“We are performing these studies in settings where nobody cares about their ranking — it does not change their pay or job security,” said Jesse Rothstein, an economist at the University of California, Berkeley, whose work criticizing other value-added assessments unions frequently cite. “But if you start to change that, there is going to be a range of responses.”

But look at this, the reporter slips Rothstein the knife, even as he is held up as the lone voice of dissent. Listen: “whose work criticizing other value-added assessments unions frequently cite.” First, I am no master prose stylist, but really? What were the alternatives that Lowery tried on that one? Union-favorite economist Jesse Rothstein . . .” No, no, no . . . whose union-supported and cited work . . . No … I guess that is the least bad way to undermine an academic economist without calling him a union shill (but still insinuating it). I noticed no one else got a “who reformy billionaires frequently cite” moniker.

Then, Lowery follows that up with… “Many other researchers…” of course implying that poor Rothstein is in lonely scientific company (but probably sitting pretty on all his union citations). Those two paragraphs seemed to me to be the reporter’s way of presenting a “there are two equal sides to the debate” but wink wink nod nod, knowhatImean, some are more equal than others.

Personal
Ok, if you’ve gotten this far, I thought I would give a little personal background of mine. I went to the DC Public Schools. I had some bad teachers. My high school psychology teacher (yes, psychology!) used textbooks that were from 1948 and were stamped “condemned.” I asked why we used these, and she said “human nature hasn’t changed.” I persisted, “But but but, our understanding of it has, right?” To no avail. I remember a question on an exam was “boys and girls are different in that they _____ and they are the same in that they _______.” There was a lot of talk show psychology presented alongside the (laughingly dated) textbook, and classes consisted of mostly uninformed musing from everybody, myself included. I challenged the teacher for the first quarter and then I gave up in disgust in the second quarter and got a C.

The 18-year-old me would probably have found much to like in these studies, but then again, I also had some great teachers. I had a calculus teacher who stayed in at lunch and afterschool, helping many kids with math, and made me want to be a mathematician (until a college math teacher beat it out of me). My computer science teacher taught me how to program in Pascal, and how to see algorithms in the world. I had a comparative government teacher who taught me that the American political system is not a fact of nature but an accident of man. I learned Spanish from kindergarten through 6th grade in my bilingual elementary school. Each grade had two teachers, one English speaking and one Spanish speaking (but often bilingual). I was given the gift of taking multiculturalism for granted by the time I was ten years old. I had a US History teacher that gave us a scavenger hunt around DC, finding little known monuments to new historical moments, and assigning a book I still haven’t forgotten called After the Fact. I could go on.

Give NCLB another ten years and it will get rid of innovative and creative elementary programs like Oyster’s. Bilingualism is nice, but it can’t be counted on to boost scores on poorly contrived reading and math tests. And those great teachers aren’t at my high school anymore. Not because Jason Kamras didn’t come along and make them rich. Not because they got tired of waiting for savior Michelle Rhee to come along and sweep their lower performing colleagues out of the way. Not because no one came and measured their VAM. No, they got worn down by the bullshit that ALL teachers had to put up with in DCPS. They got tired of being told, in so many ways, big and little, that their effort didn’t matter. I saw them often frustrated, even as I loved to learn from them.

So how do we acknowledge that good teaching matters? How do we retain and celebrate good teachers and get more of them and have fewer bad teachers?
Here’s my modest proposal:

– Provide decent working conditions for all teachers

– Prioritize improving teaching practice, not evaluating teacher quality.

– Pay all teachers a little bit more, but mostly help them do what they came to the profession to do: inspire their students to love learning.

– Respect and support a broad array of inquiry and knowledge, so that teachers can teach a rich and varied curriculum

Do this, and you’ll find bad teachers get better, good teachers want to continue teaching, and more people want to enter teaching.

Or, you could increase class sizes, increase high stakes testing, give a rare $20,000 bonus after making teachers compete with each other for salary gains based on VAM, and get going with that firing project. Let me know how that goes.