Talking about climate change #22

When science goes bad (9 minute read)

Here’s proof, if proof were needed

The evidence builds up day by day…

Roger McGough

When it comes to conversations about climate change, we often talk about ‘trusting the science’. The extent to which people trust, or don’t trust, science has a big impact on what they think about climate change – and what they do about it. It really is a matter of trust, because most of us don’t have the expertise to evaluate the quality of the science which underpins our actions. We are placing our trust in the hands of the thousands of scientists who have been working on the problem for decades, and in the scientific process.

There are many people who don’t trust science. The chances are, you know some of them. Their reasons are complex, and often stray into the world of conspiracy theories. That’s not my topic for today, but if you are interested, I looked at this issue in some detail during the Covid-19 pandemic. Here are a couple of articles. The first talks about some of the reasons behind conspiratorial beliefs (mostly related to Covid-19 but they work for climate change in many cases). The second article gives some resources to help you if this is something you are dealing with among family and friends.

Super spreaders: part two

The Turnstone: talking about vaccines #12

On the whole, I’m someone who trusts science and scientists. Although I’ve never worked as a scientist, I have a Masters degree in Biological Science and have spent my career working at the interface of science and decision-making. I have worked with many scientists. I have used scientific papers in my work for many years and I must have read many thousands of them.

Every now and again, though, something shakes my confidence. I am reminded that not all science is created equal. Not everything called ‘science’ is reliable or correct. Some of it is rubbish, and some of it is as ficticious as the mystery novel I’m working on in my spare time.

Mystery novels aside, sometimes I doubt my own ability to know whether the science I’m reading is trustworthy. I felt this way when I learned about predatory journals. These are publications which look like scientific journals, but they publish any old rubbish that looks vaguely plausible, as long as you pay them enough. I started to wonder, was I relying on sources which were unreliable? So, I spent some time learning about predatory journals, and how to spot them, so that I could have confidence in the sources I was using. I also wrote an article to share some of what I learned and help others understand more about the issue. Here’s the link if you need it.

The Turnstone: talking about vaccines #25

I had the same kind of minor crisis in confidence when I discovered ‘paper mills’. No, I’m not talking about a place which processes pine trees into loo rolls. I’m talking about groups of people who will, for a fee, create a fake scientific paper in your name, and submit it to an actual (non-predatory) scientific journal on your behalf.

Why would anyone want to do that? The performance of scientists and academics is partly measured by the number of papers they produce, as well as the status of the journal that they publish in. There’s also a very specific problem with the Chinese healthcare industry, which requires doctors with no training in research and no access to research facilities to publish papers in order to get promoted. And it’s all the better if the paper shows the benefits of traditional Chinese medicine, because Xi Jinping is a fan.

It turns out that there are hundreds of papers being churned out by paper mills. Suddenly, I felt as if I was on shaky ground. How did I know papers I was reading were real? I had learned to spot predatory publishers, but the paper mills were getting their papers published by recognised publishers – Wiley, Elsevier, even some of the Nature journals.

These examples made me realise that I needed to understand more about the grubby underbelly of science. I needed to understand what the problems were and how to spot questionable science, so that I, and you, could have confidence in my work. I also realised that we all need to know how to respond when someone quotes an example of fraud in science as a reason to doubt the existence of climate change.

So, here is what I learned.

This is the best picture I could find to represent the grubby underbelly of science (image credit: Melanie Newfield)

What is science fraud?

There are three basic types of scientific fraud. The technical terms for these are falsifcation, fabrication and plagiarism. Most of us have heard of plagiarism, which is when someone copies someone else’s work and pretends it is their own. Fabrication is also fairly clear – it’s making things up. Falsification is more subtle. It involves taking real work and manipulating it in various ways so it gives a false impression. I prefer to call it ‘fudging’, since I think most people understand that term more easily. It also avoids confusion with Karl Popper’s idea of scientific falsification, which has nothing to do with fraud (here’s a link to a video which gives a quick explanation).

There are also ‘questionable research practices’. These fall short of fraud but are still wrong, and may indicate a wider problem. One example of a questionable research practice is not obtaining the correct ethical approval and informed consent from people involved in research. This is a familiar area for me, because I’ve needed ethical approval for my research over the last few years. Other examples include not following rules for animal welfare, not declaring conflicts of interest and not giving students the appropriate support.

An example of questionable research practice comes from the laboratory of Didier Raoult. You’ve probably never heard of him, but you may be familiar with his work – he promoted the use of hydroxychloroquine for Covid-19.  Elisabeth Bik (more on her next week) raised concerns about his laboratory publishing research based on homeless people in Marseilles and people in the Congo (they weren't specific about the country involved). The research on homeless people was done over ten years and covered many different types of health research. There were seventeen different papers published, all quoting the same ethics approval number. This is definitely not the usual practice with ethics approvals, which have to be specific to the study being done. The research from the Congo also referred to a single ethical approval number – from France. The twenty three Congo studies had no approval from any of the Congo countries, nor authors from those countries.

It’s a big problem

So, how widespread are fraudulent and questionable practices? One recent study by a group of Chinese researchers gives a good overview. They did a systematic study of 42 previous papers on the subject, and found that 2.9% of scientists admitted to at least one fraudulent practice and 12.5% admitted to at least one questionable practice. When it came to what scientists thought about the actions of their peers, 16% believed that they had seen fraudulent practices among their peers, and 40% reported questionable research practices. The difference between what scientists admitted to doing and what they observed among their peers might have been because scientists were more likely to report on their colleagues than admit their own misconduct. However, it could have also been a case where multiple scientists were reporting the misconduct of a single individual.

I’ve looked at other studies too. While the numbers are different, the conclusions are similar. It’s common enough that we should be worried.

Some areas are affected more than others

Not all fields are equal when it comes to misconduct. Biomedical science and psychology seem to have more of a problem, but it’s not entirely clear to me whether this is because there is more fraud, or it is more likely to get found out in these fields. One particularly horrific example is Paolo Macchiarini, who got away with performing dangerous medical experiments on people for nearly a decade. Most of his patients ended up dying.

Dr Con Man: the rise and fall of a celebrity scientist who fooled almost everyone | Science | The Guardian

What about climate change?

Here’s one of the critical questions, since I’m writing this article to help with conversations about climate change. Are fraud and questionable practices a problem with climate science? I’ve done some digging, and while I can’t say there’s no problem, it’s clearly far less than in biomedical science.

One of the most well-known examples of supposed fraud was given the name “climategate”. Emails leaked from the University of East Anglia apparently implied that scientists were fudging the data. But were they? Despite a high level of scrutiny, it turned out that there was no evidence. Phrases were used out of context to imply fraud that wasn’t there. Here are a couple of articles about the case, one from The Guardian and one from a US-based fact-checking website.

Climategate 10 years on: what lessons have we learned? | Climate crisis | The Guardian

'Climategate' - FactCheck.org

Further digging found a climate change expert convicted of fraud, but it was financial fraud and nothing to do with climate change.

Climate change expert sentenced to 32 months for fraud, says lying was a 'rush' (nbcnews.com)

But one area of climate change does have a problem with fraud. It’s not the underlying science – again and again, that has been shown to be reliable. But there is a big problem with carbon offsets and greenwashing. This is a topic in itself, so I’m going to put it to one side and write more about it another time. But you can feel justified if you are skeptical of companies which claim to be carbon neutral. Here’s a small sample outlining some of the problems.

This little-known federal regulator could crack down on fraudulent carbon offsets | Grist

Kenya's Ogiek people being evicted for carbon credits - lawyers - BBC News

Climate Accounting Fraud - National Whistleblower Center (whistleblowers.org)

What can we do about ‘bad’ science?

This is a really hard question. The short answer, as far as I can tell, is that there is no substitute for carefully reading papers and making sure everything makes sense. Sometimes, it takes a close check of the original data to be sure, and sometimes we need researchers to replicate important experiments to be sure if a finding is real.

But that’s no help to most of us. Even for those who are experts, evaluating the quality of science outside their own field is difficult. But there are some things that non-experts can do which can help. Next week, I will take you through some tools and tips that anyone can apply. While there are no perfect solutions, there are ways we can get better at identifying suspect science.

Finally, a poem

Today’s poem is from English poet Roger McGough.

Making the change

That was all a bit depressing, so here’s a nicer underbelly from my old cat Faust (image credit: Melanie Newfield)