How much does it matter how students feel?

As I prepare my tenure portfolio, I am catching up on entering in my student evaluation data and comments into my big spreadsheet. While I don’t think student evaluations should serve as the only data by which to judge teachers, they are full of valuable information about teaching, as long as you are willing to read between the lines. For simple 15 minute reflections at the end of a semester, they are often rich with insight into how students perceive their own learning. I don’t think they are nearly as rich with data about how much students actually learned, but how the experience felt to them. Which leads me to the question in the title: if students mostly report their feelings about the class, how do these feelings relate to what and how much they learned? Like the answers to most questions about a complicated craft like teaching, my answer is that emotions do matter in the classroom, but probably not as much as you think, and it depends.

I think we can being by staking down the posts at either end of the spectrum and rejecting them. If a student hates their experience throughout and finds no meaning in it (“this was miserable, but it got me through to where I can study cool stuff”), then I would I would conclude that the course failed for that student. I see this as a possible outcome and one to be avoided. Learning is connected to emotions, and if a student disengages emotionally from a course, it is a problem. If we put our students through stress, tedium and horrible struggle, but then don’t help them feel rewarded (emotional and otherwise) at the end, students will continuously feel that school is a place to be endured, rather than engaged.

However, I think the other pole is equally toxic. If we conclude that no learning can take place unless students are perfectly comfortable, always engaged and having fun, we do a grave disservice to our students. As Dan Willingham reminds us in a chapter heading in his wonderful “Why Don’t Students Like School?” - “People are naturally curious, but we are not naturally good thinkers.” In other words, thinking is hard. If given the choice, we would rather avoid thinking. This is why, for example, this sentence, which requires you to keep a bit more in working memory, specifically devised as an example for the claim that thinking is hard, since I would never write this on purpose (ok maybe I would but I would edit it afterwards, probably) is harder to understand than the one that follows. When we avoid thinking, we save energy and get on with our lives. Unfortunately, if we spend all semester in a glorified focus group, always attendant to feelings and engagement and not to learning, the learning does not happen all by itself. Reminding students what they already know, and trying to boost their confidence does not teach them anything. As it turns out, it doesn’t even boost their self-esteem. I know that there are teachers who lead their students through emotional transformations, and I so seek to connect with students’ emotions, but I fear we can go too far in fostering student engagement and attention.

So, I am left somewhere in the middle. School is not entertainment, and sometimes steps in learning something are not enjoyable. But if students are entirely emotionally disengaged, then they probably aren’t learning either. As I try to balance the fun and the tedium, the struggle and the joy, I try to help my students use the fun to stay curious and dig deeper (“yes it’s amazing, but why does that illusion look that way? What does it mean about your eyes?”) and frame the struggle as necessary and productive. I make sports metaphors about running and lifting waits and “just getting your reps in.” But ultimately, sometimes the way that they frame their own learning is so powerful it can’t be overcome. In this way, the associated question is just as important “How important do students think their feelings are to their learning?” I think the answer to this question is that students drastically overestimate the importance of their own feelings on their learning, because they often interpret these feelings (of struggle, of discomfort, of failure) through incorrect theories of the mind.

Related to student attitudes on learning, a few comments jumped out at me recently, and reinforced to me how the myth of learning styles can have corrosive consequences. These comments inevitably begin with “I am an x learner.” Sometimes they seem positive, like “I am a visual learner, so I loved the videos and visual examples.” Other times they are negative “I prefer more discussion, so the lectures bored me” or “I prefer to learn through lecture, so the discussions didn’t work for me.”  But ultimately even the positive ones suggest the same underlying attitude. When met with challenge, they have thoughts with the same, tragic, logical form of  “I found this element challenging due to a stable preference or personality trait of mine, therefore I disengaged because there was nothing I could do.”

While some of my colleagues who are fans of progressive education could claim that this attitude is solely a consequence of the student-as-consumer mentality, in which a market model is supposed to honor every preference, no matter how trivial. But I think a second factor contributing to this disengagement with struggle is a vague egalitarianism which seeks to minimize the role of ability, but ends up backfiring. When we tell a student “Oh, don’t worry about memorizing that speech, you’re a visual learner try this instead,” we’re trying to help them find engagement, but just as often we’re helping them avoid struggle. We might be trying to avoid telling them that Johnny just has a better working memory than Billy and can memorize these things easier. But what we should be saying is this: “You might have to work a little harder, but you can still learn this just as well.” I know this isn’t exactly what the data say, but as a teacher I feel like it is better to frame ability as “time to practice to reach competence.”  We might each have a different pace in learning how to walk, talk, read and write, but (with the notable exception of certain disabilities) we can each reach competence in these tasks. We’d be far better off if more students believed the same thing about calculus and neuroscience.

So, in the end, I am happy with a few comments of disgruntled students, groaning that the course was too hard, or had too much reading. And my heart is warmed by the student who remarks “The tests are a bit hard, but I think that is how they should be” and “I liked the pass/fail grade style because it helps me learn without killing my grades.” But the most depressing comment is not the aggrieved student who feels cheated out of a few points and denigrates the whole course because of it, but the resigned one who says “class seemed good, but not for me, was useful for only certain kinds of learners.”

Posted in education, higherEd, psychology | Tagged , , , | 7 Comments

Oh I get tenure, with a little help from my friends

To continue from my last post, one of the elements that disturbed me about defining scientist as “gets grants, has groundbreaking ideas” is not just that this narrow definition of scientist excludes worthy people, but also that it excludes certain activities of people (like myself) who already consider themselves scientists. While I don’t have the “Major Research University” and the “Heavily dependent on federal grant funding” boxes filled on my “What makes you a scientist?” checklist, I do have most others. I have a tenure-track job in academia. I mentor students, I serve as a peer reviewer, I design and conduct experiments. A narrow view of science discourages people like me from doing things that “don’t count” as real science, such as science outreach, application and translation of basic research, mentoring both formal and informal, being an public intellectual on social media, being an advocate for an ethical higher education. Like this blog.

One of the ways that I work to redefine a narrow view of science is by identifying with a broad range of activities myself, both in times when it doesn’t matter that much, like describing some of my work at a cocktail party, but also times when it really matters, like describing what my activities to the tenure and promotion committee here at Randolph-Macon. This second audience is on my mind a lot lately, as my tenure portfolio is due on February 3, just a few short weeks away. I have been thinking about how to communicate the impact that I have through my online activities to someone who doesn’t use twitter or read many blogs. One way is to list these activities on my cv (Riener cv full Jan 2014), (inspired by the highly accomplished John Hawks). To do so I have to also convey the role that social media and online communication can have in conversations both within a scientific community as well as between scientists and the public.

Inspired by Bradley Voytek’s crowdsourced letter of recommendation for his first tenure track position (his original request), I am hereby requesting you, my gentle readers (except you, Mom and Dad, sorry) to help me in this task. If this blog, my other writing or tweeting, or talks or any other activities have influenced you (in a positive way), I would much appreciate an email to my gmail address: criener, or a DM on twitter, along with a signature that tells the committee a little bit of who you are. I would love to hear from a range of people and roles. While a lot of my teaching and advising here at Randolph-Macon involves deep knowledge of my students, even small touches, or snippets of communication on twitter can end up amounting to something substantial, and I want to tell that story as a part of my own development, impact and future as a teacher-scholar.

Just a word of reassurance to those who are familiar with the tenure process and worried about this as a risky move: At Randolph-Macon teaching effectiveness is weighted most heavily, with scholarship and service tied for second. My teaching has consistently been rated quite well by students as well as carefully evaluated by faculty. While I see online activities and social media as informing and informed by each of the spheres of my professional life, I will include this crowdsourced letter under service. Hopefully many of you can get a sense of the kind of dedicated teacher than I am through my writing, but this letter will not serve as a large piece of evidence of my teaching effectiveness. I have also requested the appropriate confidential letters of recommendation from colleagues at other institutions, so those will be separate.

At the beginning of this new year, I am grateful for the community of kindred spirits I have found online. As an introvert overwhelmed by most conferences, it has been so rewarding to find and help build a community of fellow scientists, teachers, scholars, or just people curious about the same stuff I am. I thank you in advance for this favor, and I hope I will be able to return it in the not too distant future.

Posted in higherEd, science | Tagged , , | 7 Comments

Tents, Tribes and Lonely Islands: Who Gets to Be a Scientist?

A recent post by thoughtful, charismatic, and talented friend Scicurious on how the “system” of science training failed her, but should have failed her sooner has gotten me thinking a lot about my role in the science “system.” Sci’s argument is that she had many early dreams of becoming a professor and scientist, but ultimately came to the realization that she just wasn’t cut out to be a scientist:

I am not cut out to be a scientist. I’m cut out to be a lot of things. A teacher, a communicator, a writer. But a grant writing, publishing, committee serving scientist? I don’t think so.

Of course Sci has landed on her feet, and is well on her way to an illustrious career in science writing (a field which she had already accomplished a great deal as a graduate student and postdoc). I’m glad to read her acceptance at coming to the realization that she did not want to write grants for the rest of her life, and that the world of big wig R1 science was not a good match for her. But it is painful to read her lingering sense of failure

And yes, I feel like a failure sometimes. Seeing other people succeed in science where I did not. I drank the academic koolaid HARD, and believed that “success” looked like a tenure track position. It doesn’t help that other people drank the koolaid, too. I have been called a failure, a quitter. I’ve been told that it’s my fault that I didn’t stay to be a role model to women in science. Every time I interact with people from my “former life”, I feel like I failed them, failed my training, failed myself. I feel like I should have worked harder, worked more, maybe not had a blog (something that has been mentioned to me many, many times) or studied harder or been more careful, somewhere.

In closing, Sci wishes that the “system” of academic science recognized that she wasn’t cut out for academia earlier and kicked her out of the science tent so she could get on to her “outside of science” career. 

To me, this is just so so sad. Because ultimately, this feeling of failure is not just bad for Sci, but it is bad for science. The more exclusive we make the tent of “Real Scientists,” the more we shrink the respect that the public has for science in general. As Janet Stemwedel aptly points out

But if the question is who counts as a member of the tribe of science, some of these factors render invisible lots of people whose knowledge, work, and interests look pretty darned scientific.

Does it cost the scientists at the top of the food chain anything to have a somewhat more inclusive view of who’s a scientist (or contributes to the well being of the scientific enterprise)? What does it cost them to keep discounting the scientists who don’t fit the narrow approved mold?

Science needs more people like Sci. Full stop. But it needs people like Sci to do the things she is great at, writing, communicating, mentoring, inspiring. The gatekeeping scientists that have told Sci she is a failure, or not a real scientist, think the currency of science should be creating new knowledge (and new, expensive, fundable knowledge, at that). What they don’t realize is that by denying the multiplicity of ways of being a scientist, in seeking to carefully guard the prestige they have so carefully amassed, they are diminishing their own status. In chipping away at their own exclusive island, they are ignoring the public sea levels of discontent with science that continue to rise. The biologist might snicker, as political science gets its entire NSF funding cut, thinking “Well, it wasn’t a real science after all.” But the biologist ignores that just because he is standing on higher ground, doesn’t mean that the logic of people like Tom Coburn will spare basic biological science. Too many legislators are happy to call biology science, but really what they want is immediately applicable medical research. Which results in idiotic statements like Sarah Palin mocking fruit fly research and real harm to basic science funding.

So here’s my challenge to Sci (and hearty defense of my own work): You ARE a scientist. Stand on that island and say “I am Science, hear me roar!” and do the things you love to do, promote science, explain science, call out shady science, etc. This too is science. If it is not we are all lost. Science will not regain public trust through careful exclusivity and identity policing.

Stay and grow the island, don’t wish you had gotten kicked off earlier. 

Posted in science | Tagged , | 11 Comments

On the Benefits of “Overreaction” – #IStandwithDNLee

First, a few of my beliefs, to clarify the provocative title and scare quotes:

Our society doesn’t react strongly enough to the subtle racism, sexism, and generic cruelty experienced by minorities in many communities.

The reaction to the specific situation with Danielle Lee is appropriate and useful. We should all be thankful, whatever our gender or race, for both Danielle for bringing it up, and her friends and community for summoning the twitterstorm. Calling out bad behavior is a necessary step on the road to a more inclusive community, and a more just society. Period.

For those of my readers who don’t know what happened (if you do, skip to the end of the summary), here’s my summary.

<Begin Summary>

Dr. Danielle Lee is an African American biologist who, in addition to being a postdoctoral researcher, is active in science writing and science outreach to urban populations. She’s a friend of mine on twitter, and I have chatted with her a few times (and attended sessions she has led), at Science Online, a conference for online science writing. She has a blog at Scientific American called “The Urban Scientist.” She was asked to write a guest blog post for a web site that curates and hosts a lot of biology content (biology-online). She asked about details and payment. Told there was no payment but “exposure,”  she politely declined. The editor responded with “are you an urban scientist or an urban whore?”

From there, Dr. Lee posted a thoughtful and reflective video and response on her blog at Scientific American. She describes being angry, but also reiterating her philosophy on being a professional science writer. This means she does not work for free, but if she believes in the cause, she may waive her fee, donating her services to a worthy cause.

There is no link there, because Scientific American removed the blog post, first citing the fact that it “verged into the personal, and Scientific American is for discovering science.” Others have pointed out that this response is BS. In a later, longer response Scientific American editor-in-chief Mariette DiChristina said it was for legal reasons. In my eyes, and in the eyes of many readers and members of their blog community, they are still digging. Despite the fact that biology-online has fired the editor and apologized to Dr. Lee, Scientific American has still not re-posted Dr. Lee’s post.

Ok, this set off a flurry of blog and twitter activity. Dr. Isis reposted Danielle’s original post. Dr. Rubidium made it clear how Scientific American’s actions made her feel as a female scientist of color. Many others weighed in and spread the word. Here is a good point from Kate Clancy on the current status of the situation, why Scientific American’s responses are still not adequate, and why this is exactly the kind of thing Scientific American should be worried about, and why policing the boundaries of acceptable science is so often fraught with racism and sexism. Here’s Dr. Isis’s last installment of thoughts. Of course, Dr. Lee’s story and Isis’ original post (“batsignal” indeed) later drew attention from more traditional web outlets such as Buzzfeed, Chronicle of Higher Education and Inside Higher Ed (although “Outrage among women? really? scientists and science writers are pretty outraged, not just women).

<end summary>

Ok. After all of this, some people are saying, “What’s the big deal?” An editor unprofessionally insulted (or, if we are in the mood to minimize “acted like a jerk”)  a person of color, and the “pc word police” set forth a storm of outrage. Others have pointed out that this kind of treatment is sadly not all that uncommon. Women and minorities deal with this kind of thoughtless cruelty all the time, why should this be different?

This reminds me of two things. First, one of my favorite papers in psychology. Second, a story and metaphor.

First, the paper is called “The Peculiar Longevity of Things Not So Bad,” by Dan Gilbert, Matthew Lieberman, Carey Morewedge and Tim Wilson. Please bear with me, I do not mean that this event is “not so bad,” but rather than events that do not trigger large responses can sometimes have larger effects than those that do. The paper posits a psychological immune system, one that helps us cope with tragic events, but also brings us back to equilibrium when we experience great joy. When we experience a tragic event, this psychological immune system goes to work, trying to make meaning of the event. We try to make sense of tragedy, find something positive about it, and try to return to our emotional baseline. This basic construct has been supported with studies of people who have undergone tragedies like the death of a loved one, or severe injuries. They change their lives, and their lives change, and they often find meaning and purpose through their adversity. In this paper, they test the idea that to engage the psychological immune system, some threshhold of pain (or pleasure) must be reached for the person to try to make sense, to find meaning and return to their baseline.

So how do they test this? They compare a situations in which you are insulted by a partner or by a non-partner. They compare being a victim of an insult to being a bystander. As it turns out, when someone is closer to the person doing the insulting, either as a direct victim, or as a partner, they see the insult as less insulting, and later feel more positive towards the perpetrator than the bystanders or non-partners.

This study fits into a broad literature, in which the psychological immune system is seen as an important way to cope with the emotional events the world throws at us. What does it have to do with #IStandWithDNLee?

I see this sort of event as the kind that would normally not trigger a “psychological immune system” response. Women, especially women of color, deal with these supposed darts of cruelty often, and simply call them “Tuesday.”

But to my eyes, this is the modern, insidious shape of prejudice. It disrupts the minds and work of scientists, scientists-to-be, and well-then-why-should-I-try-to-be-a-scientists. In snide comments and little notes, it whispers “you don’t belong here.” This event seems little, but these grains of sand make a mountain. If we keep picking them up, and as we seek to weigh them, they run through our fingers, we will never realize how they accumulate.

This stranger didn’t have any power over Dr. Lee. I assume she would have had very little legal recourse, since she had no working relationship with the editor at biology-online. She could have merely stewed, as I am sure she has before, as I am sure others have before. She wouldn’t have engaged her vast internet immune system. 

But she did. And it kicked in, did it ever. And the editor is now fired. And others should realize that random wanton cruelty to women can at least sometimes come with consequences. This is the benefit of “overreaction” I am referring to in the title. The holding up this grain of sand and saying, “Hey! This is real. This happened!” It may not be a mountain, but it is sadly representative. I hope it leads to more, not less of calling out this behavior.

One last thought for me: Why do I find this situation “useful,” as I remarked at the beginning? Because when I think of moments like this, it reminds me of when my dad and I were installing a skylight above his bedroom, must have been twenty years ago. It was a huge piece of safety glass, designed so that when it breaks, it splinters into thousands of small pieces, not big sharp shards. It was also coated with plastic so that when it broke, these small pieces would not immediately fall, but would stick in place. We were carefully pushing it into place, and we pushed just a bit too hard. We heard a little “tic,” almost like the sound of a light switch, and immediately, the entire, 60 pound, 5 foot long piece of glass was covered with a vast network of cracks. Millions of little fault lines, previously laying latent in the structure of the glass, now, instantly visible.

To many with privilege, our society seems like a clear piece of glass. We work hard, get educated, find jobs, create families, buy houses. Laws protect us. But many among us cope with invisible prejudice and discrimination. They labor through the glass, coping with acts of small cruelty, instances of invisible, unintentional, real prejudice. To them, the glass is already broken.

We should see appreciate the situation with Danielle Lee for what it is. The “tic” that reveals to us all, not that one accomplished, esteemed postdoctoral researcher can be casually insulted by a sexist who is immediately fired.  But the worlds that seem clear to the privileged have vast networks of cracks. These cracks are illustrated in Scientific American’s response, initially cautious of siding with Dr. Lee, trying to find a reason to defuse and minimize the situation, then hiding behind legal concern. What would it have done if it trusted Dr. Lee that she was not lying or deceitful in her post? Trust her, leave it up and investigate its veracity.

These moments tell us that our society is still sick from its racist past, our society is still cracked from its sexist traditions. The more often we are reminded of these facts, the better.

Posted in science | Tagged , , , , | 7 Comments

Student Learning and Labor Policies, follow up

My piece for the Atlantic ran yesterday, on how student learning is not directly connected to exploitative labor policies. I had some interesting conversations, on twitter and over email, so I thought I would share those with my readers. It is also different to work with an editor for another outlet (for a broader readership) so I’ll share some thoughts related to that.

First, one of the things that was left on the cutting room floor was a more thorough treatment and context of the Northwestern paper. I think I did a better job of that on my longer blog post on the subject, but it boils down to this: the Figlio paper compared two groups of well-paid, full benefits, long-term professors. One group is more focused on research (but also teaches), and one is more singularly focused on teaching. The article was clear, but many of the press reports obscured this fact. I should have tried to use my piece to correct the erroneous suggestion by many popular press reports that these results somehow applied to adjuncts, or indicated that tenured professors were worse (than some unspecified alternative). Edward Kazarian does just that in his piece in Inside Higher Ed, and in my mind, the Chronicle should have to syndicate it and put a note at the top of their piece headlined “Adjuncts are better teachers than tenured professors, study finds.”  Here’s Kazarian:

What thereby shifts into the background — though it does not go unmentioned — may, in fact, be the most important finding reported in the paper, that this successful cohort of “non-tenure-track faculty” were not short-term temps, but rather long-term employees. Admittedly, it was also downplayed by the study’s authors. They remind their readers that Northwestern is an elite institution, and that “its ability to attract first-class non-tenure-track faculty may be different from that of most institutions.” But the only details they give about these faculty appear in a footnote, which tells us only that “[a]lmost all classes taught by non-tenure-track faculty at Northwestern are taught by those with a longer-term relationship with the university.”

In an email exchange, David Figlio also informed me that non-tenure track teachers at Northwestern (of whom well over 80% are full-time with equal benefits) also have career ladders and ranks. In the case of the college of arts and sciences ranging from lecturer to “professor of instruction.” 

So, what is going on at Northwestern is very different from what is happening in the rest of the country. What is going on at Northwestern? It is becoming a place where research and teaching are growing further apart, even if they stay within the same walls. Northwestern believes that it can forgo damage to student learning by having two groups of (well-paid, long-term, respected) faculty, one group rewarded for research and another rewarded for teaching.  I have my doubts about this as a solution to what the paper calls “a research university’s multitasking problem,” but Figlio is not advocating that switching to a part-time adjunct labor force is no cause for alarm, or that it is educationally positive, and I shouldn’t have insinuated that.

I am still annoyed at the title of their paper (“Are Tenured Track Professors Better Teachers?”) because I don’t think this actually has anything to do with tenure. Can one provide generous salaries, institutional support and long-term job stability without tenure? Northwestern seems to be trying with its lecturers. That might be an interesting experiment, but it is not about whether tenure makes one a better teacher, but whether Northwestern’s particular definition of “non-tenure” can still yield good outcomes. And it does. Good for them.

BUT…

I still believe in my main point, which is that arguing over student learning outcomes as outputs of labor practices is short-sighted. Regardless of what the Northwestern paper shows, or what the previous research shows, I don’t think measuring student learning outcomes are the right lens to either see or change the injustice of the newly transformed higher education labor market.

A few commenters, both on twitter and at the Atlantic, argued that research showing how poor working conditions detract from student learning is a critical piece of the puzzle, and that leaving that research aside is naive. Here’s commenter Jonathan Kaplan, summing up that view well (but his entire comments are worth reading as well):

One reason that fact is important is that this study will get used to resist calls to treat adjuncts better. Why can’t we just make the basic moral argument you propose? Well, consider that there is, for example, an enormous literature on the role that air-conditioning plays in student learning. You suggest above that conducting that research was a pointless mistake — an immoral exercise in demanding that we come up with an explanation in terms of efficient teaching rather than thinking of not forcing children and teachers to sit in stifling classrooms as a basic condition of decent human behavior. But of course there is a reason people do those studies, and that is that this *wasn’t* seen as an issue of basic human decency, or if it was, no one cared enough to do anything about it.

A few people on twitter made similar points, arguing that I was taking important evidence off the table by not acknowledging that labor conditions DO matter for learning, and that a teacher’s working conditions are a student’s learning conditions. I was paraphrased as saying “Administrators, great news, you can pay adjuncts shit and it doesn’t matter!”

Jonathan Kaplan’s last sentence gives me a good place to begin my response. He maintains that doing these studies to document the harm done by lack of human decency is part of the path to getting change. I guess the argument is that showing these heartless technocrats our evidence that air conditioning is a vital ingredient to learning will convince them to fund it, since they haven’t seen it already as basic human decency.

I am a social scientist who believes in the value of social science. I am a scholar of teaching and learning who believes that measuring learning can help us better understand and improve it. But I just don’t think more social science is going to change people’s minds about this. I am extremely skeptical that more rigorous studies of student learning are going to document exactly how many “points” of learning a student loses when their teacher doesn’t have healthcare, or has to hold office hours out of their car. For this case, at least, I don’t see the path to justice as paved with data.

Jordan Weissman has a nice, long piece reviewing economic literature on labor policies and learning. He ends by responding to my piece, which would seem to contradict his (and others) insistence that more research is needed:

Hopefully, though, there will be more research. Cedar Reiner [sic], a psychology professor at Randolph Macon College, recently argued on this site that we shouldn’t think of education as a labor or economic issue. But higher ed is at an economic crossroads; the labor model is changing whether we like it or not, and it’s changing in ways that may limit the time and energy professors can devote to teaching, both in the classroom and out. We owe it to ourselves to find out whether that’s costing students.

I think gathering more data about the relationship between student learning outcomes is not going to help our argument that a mistreated professor is a worse teacher than a well-treated one. More information will not settle this score. Weissman ends with “we owe it to ourselves to find out whether that’s costing students” after he has reviewed a literature that converges on an answer: it does cost students. Of course it does. The only question is how much.

But if  we acknowledge this is indeed the question of interest, we are having the wrong conversation. How many learning points are we sacrificing by using short-term, part-time labor? Is it the short-term or the part-time that is important? How many learning points are we sacrificing for no health insurance? How many learning points are we sacrificing by paying only $2000 per course instead of $4,000? Then we argue over how to measure learning points.

I’ll end with two points: I happen to think that these are generally not well-framed social science questions. If we want to investigate the factors that change learning, we should investigate the causes that are more immediate. Pedagogical practices, curriculum, student motivation and other student factors, these are the things that truly matter for learning. Labor practices can only be mediated through how they impact the classroom.

But even if it were good social science, I am dubious of the power of research facts to move value-based mountains. There are massive amounts of social science documenting the damage caused by extreme stress or hunger in childhood. Social science has documented the cognitive impact of poverty, and the benefits of robust early childhood education. But this doesn’t stop Republicans from cutting food stamps or arguing about the value of Head Start. “You have your studies, I have mine.” Certainly we should do and recognize good research, but we should recognize that you can’t fight a value system that begins and ends with Ayn Rand with charts and data.

So what should we do? Organize. Give money to AAUP and the New Faculty Majority. Win elections (local, regional, state and national). Oppose exploitative labor practices not because they are ineffective (yes, which they probably are), but because they are wrong.

Posted in higherEd, politics, science, teaching | Tagged , , , | 3 Comments

Teaching and learning, labor and fairness

It seems a requirement that any conversation about higher education in America must begin and end with costs and economic outcomes. Along the way, our economic analysts nod to the power of knowledge (economic research shows it improves career prospects!), or the value of an educated citizenry (it produces more innovation, which of course produces more jobs, or at least more Steve Jobs’). Their implications are both personal and national: completing a college degree will get you a job, and a better, more effective and efficient higher education system will produce more jobs for us all. Unfortunately, as many new college graduates, and our country’s unemployed are now discovering, this view of the value and power of higher education is simply not true. Learning is not improved by a more efficient labor model, and inequality and injustice in our labor market are not solved with more learning. Learning is a complicated mess of students and teachers, texts and technology, content and curricula, bodies and brains. Learning is not a labor issue.

The economic mode of analysis typically begins with a blank slate. Dylan Matthews titles his ten-part series on the economics of higher education “The Tuition is too Damn High.”  In his last post on possible solutions, he addresses the relationship between cost and quality:

But it raises a troubling point: we, at the present moment, have literally no idea how good different higher education institutions are. We don’t know anything about which are better at imparting given bodies of knowledge, which are better at getting their students paying jobs, which are better at producing voters and soldiers and other contributors to civic life, or any number of other outcomes.

Because a simple and clear relationship between cost and quality doesn’t exist, it is assumed we have no idea what quality is.  Because we can’t measure quality with the same simple metrics by which we measure money, it is assumed that we have no idea what learning is. To understand the relationship between learning and money, we apparently need to measure them each by a single number, or a weighted average of a common set of metrics.

When we do find a number, we can be sure it is a bad one. Robert Gordon in the New York Times begins his lamentation about the sorry state of education in this country

For most of American history, parents could expect that their children would, on average, be much better educated than they were. But that is no longer true. This development has serious consequences for the economy.

Gordon proceeds to offer as evidence the mediocre performance of American schoolchildren on the tests from the Program for International Student Assessment and an allusion to colleges which have “longstanding problems with quality.”

A recent study by economists (and including the President of Northwestern University, also an economist) of the differences in learning outcomes compared tenure track to non-tenure track teachers. The provocatively titled “Are Tenure Track Professors Better Teachers?” asked what the consequences to student learning of a labor model in which some full time workers have tenure and some do not (despite often having long-standing relationships with the college). They found that students who had untenured faculty in introductory courses were more likely to take a more advanced course, and performed better in these courses than their counterparts who had introductory courses taught by tenured or tenure-track faculty.

Jordan Weissman, in his summary at the Atlantic notes:

Previous studies have suggested that colleges tend to hurt their graduation rates by hiring more part-time and non-tenure faculty. But Shapiro and his team wanted to measure the impact of tenure on “genuine student learning,” a notoriously tricky task.

The assessment of learning was through grades in those second courses. Figlio, the lead author on the paper, told Dan Berrett of the Chronicle of Higher Education:

Northwestern uses a four-point scale for grade-point av­er­ages, which Mr. Figlio said is a bet­ter proxy for learn­ing than stu­dent-sat­is­fac­tion sur­veys or standardized tests. “It’s not per­fect,” he said, “but frank­ly it’s the only thing I can think of.”

Berrett closes his mis-titled piece “Adjuncts make better teachers” (mistitled because the comparison group were not part-time, temporary adjuncts but rather full time lecturers) with the following quote from the paper

“Per­haps,” they wrote, “the grow­ing prac­tice of hir­ing a combina­tion of re­search-in­ten­sive ten­ure-track fac­ul­ty members and teach­ing-in­ten­sive lec­tur­ers may be an ef­fi­cient and edu­ca­tion­al­ly pos­i­tive so­lu­tion to a re­search uni­ver­si­ty’s mul­titask­ing prob­lem.”

Here you have it. A higher education labor model which splits research and teaching entirely may be efficient and educationally positive.

These economists and economic reporters treat education as an enormous, noisy and wasteful machine. We hear talk of “inputs,” “levers” and “outputs.”  We feed money into this beast and out come college graduates, ready to grow our economy. If only we could figure out how to adjust these levers, they muse, so that we had to feed less money in, and the same amount of learning would come out. Wouldn’t that be great? What if we could shift down this “labor costs” lever and see what happen to the learning outcomes?

And here is where I must strenuously object. College is not a machine (it’s made out of people! Bowling Green is made out of people!). Learning is not a labor issue.

What do I mean by this? The labor costs lever is only weakly connected to the factors that do change student learning. While education is immensely complicated, listing the important elements of a successful educational experience is not that difficult. Ask any group of college students and teachers, they’ll tell you. Motivated and prepared students. A coherent and structured curriculum. Interesting content and activities at the right cognitive level. A class size where the student feels attended to. Engaging pedagogical style. A trained and expert teacher with resources and support.

Can we affect learning by fiddling with the labor model lever? Of course, but it would be mediated through these factors. It would be cheaper to make all college classes 400 students, that will constrain the attention that can be devoted to any one of them. It would constrain the assignments and feedback offered. What if professors’ entire job consisted of teaching one class of 20 students? That would open pedagogical doors and leave more time for teacher preparation and activity design.

But ultimately, we must admit that most labor practices alone, good or bad, have a weak correlation with learning outcomes. Adjuncts and part time labor can and do provide great learning experiences. Tenured experts can provide awful ones. Charismatic lecturers can inspire in 300 person lectures, and a disorganized and disillusioned grouch can create an awkward and useless active learning discussion.  The reason is not their labor status, but whatever effect this labor status has on the other important elements of the learning situation.

The econometric study of “Are Tenured Professors Better Teachers?” would be better titled “Are Tenured Professors (at a wealthy, selective private school with motivated, traditional-aged college students, in large introductory courses,  with lecture style pedagogy) Better Teachers (than long term lecturers at the same college, who were far better paid than most adjuncts, who had taught at the college for longer than six quarters).”

But how could these lecturers be better teachers than their tenure-track counterparts? Were they selected to be better teachers whereas the tenure track were selected based on scholarship? Were they pressured to teach well, believing that their jobs were dependent on student learning and satisfaction? If they taught better, what did they do that was better? IF whatever they are doing is improving student learning outcomes, why can’t we ask tenure track professors to do that as well? This is the question of someone who is interested in learning and not merely cost efficiency. 

The modified title isn’t quite as catchy, but it illustrates how many important variables one must control for to reach a comparison of learning outcomes based on labor categories. This isn’t merely ignoring the elephants in the classroom, this is a statistically sophisticated elbowing past elephants to reach a small anteroom of teacher characteristics and incentives. The difference between these groups may be statistically significant, but it is not meaningful for understanding student learning.

As one of the people inside this machine, I can tell these macroeconomic observers that the inner operations are neither simple gears nor complex magic but something in between. Something human. While we are all rightly confused about the connection between labor practices and human learning, this doesn’t mean that practitioners have no idea how learning works. Fostering genuine student learning isn’t just “a notoriously tricky task,” but our life’s work. This isn’t simply “imparting given bodies of knowledge” but filling the pail and lighting the fire (to bend a phrase from Yeats). And are we concerned with providing economists with nationally comparable measures of student learning outcomes? Mostly not.

So if changing labor practices are not going to change student learning outcomes, why pay attention to labor practices at all? Because not merely learning outcome efficiency, but a national code of ethics should inform our labor practices. We’ve outlawed child labor not because it is inefficient, but because it is exploitative. We should consider the same when we see that higher education is now 76% part time labor. For all those statistics that economists such as Gordon begin with the importance of education, what about the importance of health benefits and employment stability? A labor force that is 90% part time might just fight to “produce” as much student learning as the current one (76% part time) in the coarse ways that economists measure student learning. But that wouldn’t make it right.

So here’s my prescription for an improvement in our national dialogue on education. Reporters, do you care about student learning? Then report on the things that actually matter for student learning. Learn a bit about pedagogy. What is the curriculum? What is the content? Read some of the people who care about student outcomes beyond how much they make, at the National Institute of Learning Outcomes Assessment.  Consider the research and guidelines of the American Association of Colleges and Universities in their Liberal Education and America’s Promise (LEAP) Program, in which they offer a vision of learning (and its measurement) that goes beyond standardized testing and beyond the only thing that Mr. Figlio can think of.

And to my fellow higher education faculty, tenured or not: don’t take the bait. Don’t criticize the details of a new study of the impact of teacher pay or tenure on student learning. A labor model is not a learning model. The best thing that a labor model can do is take care of food, clothing, security and health, and let teachers and students focus on learning. The issues of labor in higher education are issues of fairness, not efficiency.

Posted in education, higherEd, politics, teaching | Tagged , , , , | 4 Comments

Simple, general solutions to college problems (just add data)

“Get some burgers, get some beers data, a few laughs, Dude, our troubles are over.”

http://www.metacafe.com/watch/an-KDCB4bbmbh24m/the_big_lebowski_1998_walters_plan/

I thought of Walter’s line in the Big Lebowski while reading Dylan Matthews piece in the Washington Post wonkblog on college costs (part X: How do we fix it?) this morning. Matthews seems to think that better data on student outcomes, combined with a few readjustments of federal incentives will drive down college costs and tuition. I am not as well acquainted with college costs and financial aid as Matt Reed, Sara Goldrick Raab or Sherman Dorn. I do know enough to know that it is way more complicated than most pundits realize. I often invoke Archibald and Feldman for linking higher ed costs to the trajectory of costs in the rest of the service economy. Certainly higher education does have some differences from other services that require lots of education, like dentistry. But it is far more complicated that Matthews realizes. Matt Reed makes that point quite eloquently here in response to part VI in Matthews series. But in this most recent installment, Matthews steps on an area that I do know a little more about: assessment. While Walter tells the Dude that “we’ll go out there, brace the kid, should be a pushover, we’ll get our million dollars back,” Matthews has a similarly unreasonable expectation of how easy it should be to measure student learning outcomes:

The worry with proposals like the super Pell Grant or student loan caps, as with all price controls, is that it risks damaging the quality of the institutions. That’s a valid concern. But it raises a troubling point: we, at the present moment, have literally no idea how good different higher education institutions are. <emphasis mine> We don’t know anything about which are better at imparting given bodies of knowledge, which are better at getting their students paying jobs, which are better at producing voters and soldiers and other contributors to civic life, or any number of other outcomes.

This is a huge problem with how we regard higher education. When Matthews writes “how good” he is imagining one Platonic test that compares schools on “imparting given bodies of knowledge”  and perhaps another for “contributors to civic life” or any other [finite] number of [interval scale, ranked] indices for student outcomes. He then goes on to describe NSSE data, and CLA data, and how it is such a shame that it is secret, because it is the best data we have. 

I am no data nihilist. In fact, I am the chair of my college’s assessment committee. I am curious and devoted to measuring learning outcomes in my own classes, and helping others in my college use measurement to be reflective about the curricula and pedagogy, and improve it. But the more experience I get with grappling with doing responsible assessment but leaving space for teacher creativity and student transformation, the more skeptical I am of the CLA and other national standardized tests of critical thinking skills, writing skills, etc. I get the feeling that Matthews and his ilk think that we at colleges are looking at a big bowl of ingredients, and if we just mixed it properly we would get a uniform dough that could be measured and weighed properly (“but we could be sophisticated about it, we wouldn’t just weigh it, but we could test plasiticity, or moistness”). But instead, we have the ingredients for a many different meals, and there is value to keeping dimensions separate, and incomparable. Which is better kale with olives or a red velvet cupcake? Which is better, arts education or history? Which is better, a college with a tradition of local service projects or one with great graduate school placement rates?

Saying that we have “literally no idea how good different institutions of higher education are” is like saying we have literally no idea how good restaurants are. People enter restaurants with different values and expectations, not just expecting calorie counts and one dimension of taste. And sometimes they know that a better one is across town but they feel a lot more comfortable closer to home, so they’ll deal with that rude waiter.

Matthews closes his piece with placing all his bets on better data (which he assumes starts with the CLA and NSSE):

Without better data, there’s no way to defend the contribution that college makes to our economy and our society, and no way to make that benefit cheaper for those who need it.

I love data. I’m all for better data. But let’s not confuse data with value. Most people don’t. We won’t get any closer to solving our problems by insisting people act more like economists. Maybe we’d do better if we started insisting that economists act more like people.

Posted in science | Tagged , , , | 4 Comments