Study suggests we overstate our negative feelings in surveys

From the New York University press release:

We tend to overstate our negative feelings and symptoms in surveys, shows a new study by a team of psychology researchers. This bias wears off over time, but the results point to the possibility that measurements of health and well-being, which are vital in making medical assessments and in guiding health-related research, may be misinterpreted.

“Understanding the magnitude of this bias is essential in accurately interpreting survey results that include subjective reports of feelings and symptoms,” says Patrick Shrout, a professor in New York University’s Department of Psychology and a co-author of the paper, which appears in the journal Proceedings of the National Academy of Sciences (PNAS).

It’s long been understood that survey instruments are imperfect measurements of mood and emotions. However, they nonetheless provide insights into people’s preferences, fears, and priorities — information on which policy makers, industry leaders, and health-care professionals rely in their decision-making.

Less clear, in particular, is the accuracy of capturing our sentiments over time using repeated measurements, which is a common method to gauge changes in symptoms, attitudes, and well-being. Notably, there have been puzzling findings in the psychological literature that reports of anxiety, depression, and physical symptoms decline over time, regardless of the circumstances of the people being studied.

To study this decline, the researchers conducted four separate experiments in which the subjects in each were asked, multiple times, about their anxiety, physical symptoms, and energy level.

In three of the four studies, the subjects were facing stressful events and the expectation was that anxiety and physical complaints, such as headaches and sleep disturbance, would be more common as the event drew near. One of these studies focused on recent law school graduates preparing for the bar examination and two others centered on college students who were preparing for difficult pre-med science examinations. The fourth study was a bi-monthly survey of college students over the course of an academic year. All four studies were designed so that groups of subjects gave their first reports at different times relative to the stressful event or academic year.

In all studies, the subjects reported more anxiety and symptoms the first time they completed the survey compared to their own later reports. This initial elevation was limited to the first survey day, and it was inconsistent with the course of anxiety and symptoms normally associated with a difficult event.Although previous researchers have assumed the pattern of decline was due to response bias of later reports, Shrout and his colleagues concluded that the pattern of decline over time was likely due to an overstatement of distress and symptoms the first time — rather than an understatement in later times. This is the only explanation that accounted for the fact that anxiety was more elevated four weeks before the exam than three weeks before, they note. Moreover, the law school graduates who were asked to report current anxiety and symptoms for the first time one week after the bar exam had elevation similar to others who had not yet taken the exam.

“Because many political and health surveys only ask questions one time, the bias associated with the initial overstatement can be confused with levels of genuine negativity,” observes Shrout.

The study also included researchers from Scotland’s University of Aberdeen, Purdue University, Adelphia University, UCLA, Iowa State University, Arizona State University, the University of Texas at Austin, and Columbia University.

The research was supported by grants from the National Institutes of Mental Health (R01 MH060366) and the National Institute on Alcohol Abuse and Alcoholism (R01 AA017672).