I recently read a little back and forth on the Scientific American blogs, and it reminded me of a few pet peeves I have of science writing and how the public understands science (in particular psychology) in general. Thought I would share.
To get up to speed: a recent article by Miriam Law Smith et al in the journal Hormones and Behavior found a correlation between estrogen levels and how many children college women said that they would prefer eventually having. They also found that if they took photos of all the women, and averaged them, a photo average of high maternal women (again, self-reported desire for more children) was rated as more feminine compared it to a photo average of low maternal women.
Ok. So, both Kate Clancy and Scicurious over at Scientific American blogs wrote blog posts on this article (Scicurious, Clancy) finding fault with it. Then, the author of the article (Miriam Law Smith) wrote a response also posted at Scientific American. I urge you to read them now. Or you could wait to see if you understand the rest of my post, which I think you will, but then read them to check me, and for more depth and details that I am sure I have missed.
The first thing that strikes me is that evolutionary psych often gets accused of boiling down a complex human behavior or characteristic into one silly, limited, convenient way of measuring it. Whether it is attractiveness, or maternal desire, or whatever, critics charge that this complicated facet of human nature cannot be reduced to a mere checkbox on a form. But this is a criticism of all science. We measure things. That’s what we do. Whether it is the speed of light, the curvature of a finches beak, or happiness, we take a complex interesting phenomena and turn it into numbers. When we do that, we lose variation, we lose complexity, we lose context. We lose the essence of that thing.
But here’s the rub. Scientists are not essentialists. We never were looking for the essence. I can’t be bothered that I am losing the essence of what mood someone is in when I have them fill out my questionnaire. I have my measurement, which I think is reasonably ok, in a convenient, I-have-to-finish-grad-school kind of way. I try to find other measurements to back me up (like other people who used that same questionnaire many times before). If I find that my measurement of mood can help me predict another measurement (like, how steep is that hill? Yes. that’s my dissertation in a nutshell) then good for me. But no measurement is perfect. People who study personality can’t be paralyzed by the fact that people are affected by the situation, and no single test captures what kind of a person somebody really is. We try to improve our measurements, and make others, but when someone argues that we aren’t _really_ measuring the true thing, we don’t really pay that much attention, unless they have a better measurement in mind, or even better, a study where someone used a better measurement.
To the blog posts:
Clancy’s is titled: “Framing and definitions: Are you maternal enough to be a woman?”
This reflects an essentialist framing of the study. No one is saying what it truly means to “be” a woman. Instead, the researchers are taking a shortcut, by asking 18 people to rate faces as “feminine” or “not as feminine.” Elsewhere this logic is continued. One quote at the end “Not wanting a baby today, or any day, does not make you less feminine.” No, but it may coincide with lower estrogen, and that may coincide with your face being rated as less feminine.
Scicurious admits the article made her angry. A sentence in the middle ties her anger to essentialism: “Scientists have always been interested in what makes women women, and what makes men men.” I would argue that this is not what scientists have always been interested in. Psychological scientists are interested in why people behave the way they do. We are not interested in the essence of woman or the essence of man. The question of the original article is not why are some women real women and others not, but rather, can we explain some of the variation in maternal desire using hormone levels?
I can see how this is contrary to people’s values, and it has echoes of a history (and sadly, a present) of reducing women’s agency. Men have free will, but women are simply fragile hormonal puppets. I don’t condone this sort of thinking, and I think it horribly wrong, as I imagine, does the female lead author of the study.
But this is part of what psychology (and science) often does. By illuminating mechanism, we reduce everyone’s agency. Skinner and the Behaviorists went a bit overboard, but more modern research shows that whether it is nature, nurture, or a combination (hint: it’s always a combination) we don’t have nearly as much free choice as we think we do.
Second theme: “Peer” review and context
Part of the criticism from both scicurious as well as Clancy was that the paper lacked the proper context by not acknowledging the myriad different factors that contribute to the authors (yes, convenient, limited) measure of maternal desire. Biocultural factors such as age, class, ethnicity etc obviously have a huge effect on something such as maternal desire, as well as what number college women write on a form that says “how many children do you want, ideally” whether or not that actually reflects true maternal desire. But the article did not address that to the bloggers’ satisfaction.
Here is where I see the limitation of peer review by online discussion. For a scientist in that field who is reading that article, I would submit that the data and analysis provide a fair amount of context and limitation. Others on twitter and in the comments asserted that an R2 of 0.19 was small and somehow barely worth a mention. As Law Smith points out, this means that 81% of the variation is unaccounted for. What she doesn’t add is that an r of .49 and an R2 of 0.19 is absurdly high in the context of other psychology findings. Accounting for 20% of the variation in some complex behavior is amazingly, suspiciously high. I would think that even Law Smith would think that to be at the upper end of what you would find if you did ten similar studies. But this is something you can only know if you have read a number of the other studies in that literature, and can interpret the data in that context.
But often when I read science writers criticizing articles like this (especially in evo psych), without any expertise in that literature, I find that the criticisms ring hollow because they are not of that particular article, but blanket dismissals of entire fields, without understanding the limits as well as the strengths of that field. In science, quite often the only acceptable criticisms of measurements are other measurements. If you think my way of measuring happiness doesn’t _really_ measure happiness, find a better one. But if my measurement of happiness predicts how long people live, or how happy their friends think they are, or how many times they go to the doctor, then your argument starts to lose its persuasiveness. If I am not measuring happiness, I am at least measuring something that seems to correlate with other things that we care about.
I’ll close with scicurious last paragraph:
So I wonder if the authors should make more effort to look into sociological factors. How does the intense pressure on women to become wives and mothers change as a function of how feminine the girl looks? I think you can’t separate any of this from this whole “women with higher estrogen want to be mothers” idea. This is why papers like this bug me, because they try to sell this as a evolutionary thing, without really acknowledging how much sociological pressure goes in to making women want to be mothers. And of course now I read them and I instantly get bristly, because what I see is people making assumptions about what I want, and what I must feel like, based on a few aspects of my physiology. It can be of value scientifically…but I don’t want it to apply to ME. I know it might be science, but I also find it more than a bit insulting.
The article is not making assumptions about what she wants or how she feels based on her physiology, it is saying that a physiological factor can predict some variation in behavior. I can understand how this would make her uncomfortable. I look at some findings that make me uncomfortable too, but this is not insulting, this is just the particular kind of psychology published in Hormones and Behavior. Telling the authors that they should become sociologists is not a solution to discomfort. If you feel that sociological factors totally overwhelm and explain all of the variation in maternal desire, then find a way to measure those sociological factors and contrast them to physiological ones (and maybe a better more rigorous way of measuring maternal desire). Or, more realistically for a science writer, find another study that shows this to give context.
But don’t criticize an article in hormones and behavior for being about, well, hormones and behavior.