When New York University psychologist Jonathan Haidt asked about a thousand attendees at the annual meeting of the Society for Personality and Social Psychology in 2011 to identify their political views with a show of hands, only three hands went up for “conservative or on the right.” Separately, a survey of more than 500 social and personality psychologists published in 2012 found that only 6 percent identified as conservative overall, though there was more diversity on economic and foreign policy issues.1 The survey also found that 37.5 percent of respondents expressed a willingness to discriminate against conservative colleagues when making hiring decisions. Psychologists, it appears, tend to fall on the liberal end of the political spectrum.
Social psychology’s left tilt has been widely discussed, yet it has been difficult to measure how political leanings influence the work that the field produces. But a new study has tried to quantify just that, and it found that social psychologists assess conservatives differently than liberals. It also found that scientists were aware of the potential for problems and willing to acknowledge them. The results confirm that a lack of political diversity within psychology may bias its findings on political issues.
The new study, by an international team of researchers, will be published in a forthcoming issue of the Journal of Experimental Social Psychology and had two parts.
In part one, the researchers presented 2,560 participants2 with 306 abstracts related to political beliefs or behavior drawn from 10 years’ worth of Society for Personality and Social Psychology meetings. Raters were asked to assess how the research characterized political conservatives and political liberals. They were also asked the extent to which conservatives were the target of “explanation.”
Suppose, for instance, a study finds that conservatives are less likely to change their opinions on moral issues than liberals are when exposed to counterarguments. “The researchers could explain this as ‘conservatives are cognitively rigid, inflexible, and resistant to new arguments,’ ” said Eric Luis Uhlmann, a psychologist at INSEAD in Singapore and the study’s corresponding author. “However, they could just as easily have interpreted this as ‘liberals are wishy-washy, overly flexible, and don’t stand by their principles.’ ” Uhlmann and his colleagues asked participants to rate whether a study’s findings were equally discussed in relation to liberals and conservatives, or instead were pinned on one group over the other.
Sure enough, the abstracts more often explained their findings in terms of conservative ideas rather than liberal ones, and conservatives were described more negatively in the eyes of the raters.
The effect sizes they found were “not huge,” Uhlmann said, but they were present. “For a randomly chosen abstract there’s about a 60 percent chance of it describing liberals more favorably than conservatives, and a 56 percent chance of it explaining conservatives more,” he said. (If there were no difference, you’d expect both numbers to be 50 percent.)
These differences are “statistically small, but the practical significance is potentially high,” Uhlmann said. Most effect sizes in social psychology are quite small, he said, but when they occur across society (or an entire field), their consequences can add up.
The researchers hypothesized that conservative raters might perceive a bias against them that liberals didn’t, but that wasn’t the case. Raters were asked to report their own political views and, if anything, conservative raters viewed the abstracts as less biased against conservatives than liberal ones did, Uhlmann said.
The second part of the study tested to see whether psychologists were already aware of their field’s bias. The researchers recruited 198 scientists, explained to them the study’s first part and asked them to predict the results. The scientists correctly predicted that the abstracts would be less favorable toward conservatives, but they estimated that this effect would be larger than it really was.
While it’s tempting to read into that result — psychologists think they’re even more biased than they really are! — this overestimation might be explained by the study’s sample, Uhlmann said. The researchers recruited participants through social media, and “it might be that people who believe that research is more biased were more drawn to participate,” he said.
Before making their predictions, the forecasters had been asked whether they thought social psychology evaluates conservatives and liberals differently and whether the field sought to explain the political beliefs to different degrees. When presented with the actual findings, these researchers updated their beliefs accordingly. That’s encouraging, Uhlmann said, because it provides evidence that challenging people to reconsider their beliefs when presented with contradictory evidence may be one way to create movement on scientific controversies.
“This is a very carefully conducted study, with particular care taken to ensure the results are robust across different ways one could slice the data,” said Linda Skitka, a psychologist at the University of Illinois at Chicago who was not involved in the study. Alice Eagly, a psychologist at Northwestern University, called it “a good demonstration of some bias” but cautioned that “it’s a very narrow demonstration” that focused on a single type of bias: in-group bias, the tendency to favor one’s own group. “This must not be portrayed as a big demonstration that psychology has a liberal bias, because it’s a very, very narrow demonstration of how people think of liberals and how they think of conservatives,” Eagly said.
One way to fight political bias is to work on trans-ideological teams, said Philip Tetlock, a psychologist at the University of Pennsylvania. He said he would like to see more adversarial collaborations, a kind of study design proposed by Daniel Kahneman in which researchers who disagree on an issue come to an agreement on the terms under which they would be willing to change their minds, then set ground rules on collecting data to help settle the issue. Of course, such an approach could be hard to accomplish on any scale if conservative psychologists remain a rarity.
The fact that researchers are engaging in self-reflection on this issue could be a reason for optimism. “Psychology as a field has problems, but it is not in denial. It is working on them,” Haidt said. “I have been raising the alarm and talking about the underrepresentation of conservatives since 2011, and nothing bad has happened to me — I have not been ostracized, and overall the field has reacted very positively.” Discussing an issue and studying it aren’t the same as fixing the problem, but it’s a necessary first step. The new study is “a very good conversation starter — that’s how I’d characterize it,” Tetlock said. “I hope that people build on it and extend it.”