Five Things to Check when Reading a Science News Story

Is there more than meets the eye? Frances Vaughan has some guidance on how to separate fact from fiction when reading about science in the news.

For many of us, newspapers and news websites are our primary sources of information about new scientific research. While these are great resources, it can sometimes be difficult to ascertain whether a science news story is impartial. This is particularly true when it comes to the reporting of research related to human health or disease. Before jumping to conclusions about the benefits of wine or the dangers of jogging, it’s important to ask ourselves five key questions:

1. Who?

One of the first things a science news article will state is the name of the researchers who performed the work being reported. Some helpful things to check about these researchers include whether or not they are leading figures in their field, and whether they have a history of championing a particular stance on a research topic.

2. What?

Once we know who performed the research, the next things to clarify are what they did, what they found, and what conclusions they have drawn from these findings. Similarly, it is important to check what words are being used by the journalist to describe these findings and conclusions. Are they stating that something “does” or “will” have a certain effect, or do we see more “may” and “could”? Before we change our diets or lifestyles in response to a new report, we should make sure to understand exactly what is being claimed, and the level of certainty with which these claims are being asserted.

3. Where?

As well as knowing where the research was performed, it can be helpful to know where the results of the research have been published. Typically, scientific research will be published in academic journals, but not all journals are equal. For every field of research, there are some journals which are particularly well respected, and others which fall somewhat lower down the list. Journals are currently ‘ranked’ in esteem according to their impact factor, a figure between 1 and 162.5 which is derived from the number of times the work published in that journal is cited in other articles. Although this is not a perfect system, it can be useful to check the name and impact factor of the journal in which a piece of research has been published before we decide to trust the results.

4. When?

Most journalists will report on scientific research that has been published recently. But considering how many articles are published each week, news reporters will likely have to sieve through multiple press releases before choosing one to write about. With this in mind, it is important to think about whether certain political situations or current affairs may be influencing the choice of research being reported. Come January, we are inundated with stories on how to be healthier. Come summer, we are warned of the risks of too much sun exposure. Of course, this doesn’t mean that we should be sceptical of all reporting, but it is helpful to factor the timing of a news story into our assessment of its claims.

5. Why?

Finally, understanding the motivation behind the research that is being reported, as well as the motivation for reporting it, is key to judging the validity of any conclusions drawn from the work. A quick search for the names of the lead researchers should find their academic ‘biography’ on the website of the university or research institute at which they are based. These profiles are handy for gauging the general theme of the researcher’s work, and for assessing whether there is likely to be any significant bias in the interpretation of their research findings. For example, are they being funded by an organisation with a vested interest in these results? Or is their area of interest particularly contentious? Armed with all of this knowledge, we will be in a significantly better position to judge the claims made by reporters of scientific research — which could save us from making any rash decisions about how fast we jog, or how much wine we consume!