Talking about climate change #23

What can we do about suspect science? (7 minute read)

son of the earth,

who was shaped from her,

enslaved to arrogance,

money...

Márcia Wayna Kambeba, translated by Tiffany Higgins

A couple of weeks ago, I frightened myself learning about the extent of science fraud and misconduct. I realised that the problem was much bigger than I had previously thought. I wondered whether there was anything I could do to have more confidence in the papers I was reading and using for my work.

The short answer, unfortunately, is that there is no single, simple solution. From what I have read, many cases of misconduct come to light only when someone who is part of a research laboratory raises concerns about a colleague. But there are things that people who read and use scientific papers can do. I have found some tools and resources that I am now using, which might be useful to others.

Here’s some of what I am now routinely doing so that I can have more confidence in what I’m reading.

Tip number one: identify retracted papers

Sometimes scientific papers are ‘retracted’. This means that either the authors or the journal editors decided that there were significant problems with the paper, and withdrew it. The problems weren’t necessarily fraud – they might have been simple errors not detected by the authors or peer reviewers. However, my reading suggests that some form of misconduct is a common reason for retraction of papers (for transparency, here are links to some papers on the subject: 2012 paper, 2019 paper, 2021 paper).

When a paper is retracted, it will say so on the publisher’s website. As the following examples show, this can be very obvious, or rather less so.

A critical assessment of extreme events trends in times of global warming

Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children

However, there is a catch. The paper may still be available elsewhere on the internet, without its retraction notice. This is more likely when the authors were unhappy about the retraction, or the paper represented a contrarian view on a controversial topic like climate change or vaccines. It didn’t take me long to find alternative versions of the papers above on other websites, with no information to suggest they have been retracted. Here they are – and looking at them you wouldn’t know about the retraction.

A critical assessment of extreme events trends in times of global warming

Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children

This is a big problem for someone like me. I don’t work for a scientific institute or university, so I don’t have access to a library and journal subscriptions. I often resort to finding papers on other websites. How could I know whether a paper was retracted if I found it on another site?

The obvious answer is to make sure I check the publisher’s website first, and that is something I usually do anyway. But I discovered that there was an easier way – something I was already doing.

Those of us who use a lot of scientific papers use software tools to manage those lists of papers. I use a tool called Zotero, which I admit I chose largely because it is free (also, it had good online reviews). Zotero has partnered with the website Retraction Watch, which maintains a database of papers retracted by authors and journals. When I use a scientific paper for my work, I download it to Zotero. And if it has been retracted, Zotero will tell me. I’ll get a big red banner at the top of the page warning me that the paper has been retracted.

Here’s the site to download Zotero if you want to use it.

Zotero | Your personal research assistant

The brief moment when you can see a hippo, before it disappears under the water again (image credit: Melanie Newfield)

Tip number two: spot when other scientists have doubts and questions

Nobody loves correcting a scientist more than another scientist. The whole process of science depends on scientists scrutinising each other’s work. At times this doesn’t work as well as it should, which is why I’m writing this article. Nonetheless, the peer review process has great value.

Typically, scientific peer review happens before a paper is published. However, if a scientist has questions about a published paper, they still have ways of raising these questions. One widely-used method is to post the question on a site called Pubpeer. Here, scientists raise questions that they have about published papers and in many cases the authors respond. The discussions tend to be highly technical and I don’t pretend that I can understand a lot of what is discussed there. However, it is useful to know whether questions have been raised on a paper, what kind of questions they are and how the authors have responded, if at all. I’ve included one of the more comprehensible examples here.

PubPeer - Effects of Low-Level Laser Therapy in Autism Spectrum Disorder

Fortunately, it isn’t necessary to look up every paper on Pubpeer to find out if there have been any comments. Pubpeer can be integrated with web browsers like Edge and Chrome. Download the software tool, and if you access a paper which has Pubpeer comments, you’ll get a green bar at the top of the page linking you to the Pubpeer comments. This is what I have done and I’ve found it very useful. Here is the link:

PubPeer - Search publications and join the conversation

It’s important to note that a listing on Pubpeer doesn’t mean that there is any fraud or misconduct. Many of the comments are for genuine errors, such as a typo on a mathematical formula. And the criticisms may not be valid at all, they might just be someone disagreeing with the conclusion. Nonetheless, Pubpeer is a valuable resource for those who read a lot of papers.

Acacia at Lake Elementaita (image credit: Melanie Newfield)

Tip number three: follow the work of science fraud detectives

Some people and organisations are doing great work in uncovering fraud, questionable practices and errors. I’ve learned a lot from reading articles on their websites about how to identify potentially questionable papers. I’ve listed three websites here – one from an organisation and two from individuals.

Retraction Watch

The group Retraction Watch works under the umbrella of a not-for-profit organisation called the Center for Scientific Integrity. Their website contains a number of resources, including the Retraction Watch database of retracted papers, which I mentioned is linked to Zotero. There’s also a leaderboard for authors with the most retractions, a list of the most-cited retracted papers and a blog outlining various issues with scientific integrity.

Here’s an example of one of their articles, about apparently non-existent authors.

What’s in a name? Made-up authors are penning dozens of papers – Retraction Watch

For Better Science

The site ‘For Better Science’ collects news of science fraud and other misconduct. It’s a little alarming to be honest, because there are a lot of stories. I’ve included a couple of particularly interesting examples.

Here’s one of their articles on the problem of paper mills, featuring two science fraud sleuths known online as Smut Clyde and Tiger BB8. Smut Clyde is a New Zealander, while Tiger BB8 is originally from China, now a US resident.

The Chinese Paper Mill Industry: Interview with Smut Clyde and Tiger BB8 – For Better Science

Here’s another case from Smut Clyde, this time from the field of nanotechnology research.

Zombie fingers inside corroded nano-piecrusts – For Better Science

Elisabeth Bik

Another dedicated sleuth is the Dutch microbiologist Elisabeth Bik, who runs a blog called the Science Integrity Digest. Her work has resulted in more than 1000 papers being retracted and another 1000 being corrected.

Here’s an example of an article, which talks about the retraction of hundreds of papers proven to come from paper mills.

Hindawi’s mass retraction of “Special Issues” papers – Science Integrity Digest

Here’s another article, about some questionable practices in the way that authors cite some papers.

Citation Statistics and Citation Rings – Science Integrity Digest

And a poem…

Today’s poem gave me goosebumps. I’m sorry, it won’t cheer you up, but it’s beautiful. It comes from indigenous Brazilian poet and geographer Márcia Wayna Kambeba, translated by Tiffany Higgins.

The Time of Climate, by Márcia Wayna Kambeba | World Literature Today

When I went searching online to figure out what these birds were, I discovered that they are a hybrid of two different lovebird species which are native to two different parts of Tanzania. They are widespread in Kenya (image credit: Melanie Newfield)