From Molecular Brain, Biomedical Central

Molecular Brain volume 13, Article number: 24 (2020) Cite this article

Abstract

A reproducibility crisis is a situation where many scientific studies cannot be reproduced. Inappropriate practices of science, such as HARKing, p-hacking, and selective reporting of positive results, have been suggested as causes of irreproducibility. In this editorial, I propose that a lack of raw data or data fabrication is another possible cause of irreproducibility.

As an Editor-in-Chief of Molecular Brain, I have handled 180 manuscripts since early 2017 and have made 41 editorial decisions categorized as “Revise before review,” requesting that the authors provide raw data. Surprisingly, among those 41 manuscripts, 21 were withdrawn without providing raw data, indicating that requiring raw data drove away more than half of the manuscripts. I rejected 19 out of the remaining 20 manuscripts because of insufficient raw data. Thus, more than 97% of the 41 manuscripts did not present the raw data supporting their results when requested by an editor, suggesting a possibility that the raw data did not exist from the beginning, at least in some portions of these cases.

Considering that any scientific study should be based on raw data, and that data storage space should no longer be a challenge, journals, in principle, should try to have their authors publicize raw data in a public database or journal site upon the publication of the paper to increase reproducibility of the published results and to increase public trust in science.

Introduction

The reproducibility or replicability crisis is a serious issue in which many scientific studies are difficult to reproduce or replicate. It is reported that, in the field of cancer research, only about 20–25% [1] or 11% [2] of published studies could be validated or reproduced, and that only about 36% were reproduced in the field of psychology [3]. Inappropriate practices of science, such as HARKing (Hypothesizing After the Results are Known) [4], p-hacking [5], selective reporting of positive results and poor research design [6,7,8], have been proposed to be a cause of such irreproducibility. Here, I argue that a lack of raw data is another serious possible cause of irreproducibility, by showing the results of analyses on the manuscripts that I have handled over the last 2 years for Molecular Brain. The analysis shows that many researchers did not provide the raw data, suggesting that raw data may not exist in some cases and that the lack of data may constitute a non-negligible part of the causes of the reproducibility crisis [9]. In this editorial, I argue that making raw data openly available is not only important for reuse and data mining but also for simply confirming that the results presented in the paper are truly based on actual data. With such concept, the data sharing policy of Molecular Brain has been changed and I introduce this update.

Raw data rarely comes out

As Editor-in-Chief of the journal, I have handled 180 manuscripts since early 2017 to September 2019 and have made 41 editorial decisions categorized as ‘Revise before review’, with comments asking the authors to provide raw data (Fig. 1; See Additional file 2: Table S1 for details).

Fig. 1

Flowchart of the manuscripts handled by Tsuyoshi Miyakawa in Molecular Brain from December 2017 to September 2019

Full paper/article here.

From Polar Bear Science

Worth watching if you haven’t seen it – and a second look if you have – a rare balanced documentary produced by the CBC in 2014 on polar bear conservation, with interviews with biologists Mitch Taylor and Andrew Derocher.

“In The Politics of Polar Bears, Reg Sherren will pick his way through the message track to help you decide what is really happening with the largest land carnivore on the planet.”

Short version here (about 18 minutes):

Entire version (45:30):

https://www.cbc.ca/player/play/2499432823

Online summary by the producer of the film, Reg Sherren (see excerpt below).

The most up-to-date discussion of polar bear numbers and the politics of polar bears are in my popular new book, The Polar Bear Catastrophe That Never Happened.

From Reg Sherren’s (2 September 2014) summary article:

“For some time now the suggestion has been that polar bears are in trouble and that many sub-populations of Ursus maritimus are decreasing, making them an iconic symbol in the fight against global climate change.

But there remains an ongoing debate within the scientific community that studies polar bears and their populations about whether the narrative of declining numbers is a stark reality or convenient myth.

Andrew Derocher, a professor of biological sciences at the University of Alberta, has spent his decades-long career studying polar bears, and has been more outspoken than most about the peril the big bear may face in the coming years.

“Our estimation is that we probably won’t have polar bears in Churchill once we get out to mid-century … They could be gone in a couple of years.”

“Our estimations are, if we had a very early melt, and a very late freeze, we could see up to 50 per cent mortality in a single year. You put a couple of years like that back-to-back, and things could happen very quickly,” says Derocher, in reference to a worst-case scenario about certain sub-populations he has studied. [SJC – see my post here on that issue]

…

But not everyone agrees polar bears are in trouble. Biologist Mitchell Taylor has studied polar bears and advised governments for more than thirty years, living in the high Arctic for much of that time.

“They’ve certainly been around through the last interglacial period,” says Taylor. “During that interglacial it was warmer than it is now: we had pine trees on Baffin Island, deciduous forests north of the Arctic Circle. Polar bears had to have survived that or we wouldn’t be seeing polar bears now,” he says.

Taylor asserts that polar bear populations “don’t appear to be declining” in any group that he is “aware of so far,” and that the science of estimating polar bear numbers has never been precise. He says that many of the current estimates are based upon a lacking methodology, admitting that some of his previous work incorporated the allegedly faulty technique as well.

Taylor says the problem lies in the way population estimates are extrapolated from samples.

“When you don’t sample the whole area you underestimate survival, you underestimate population numbers, and in fact the culmination of those biases can result in a scientific estimate that suggests a decline when none exists.” [SJC – See my post here about that issue]

It was just over a decade ago, says Taylor, that the notion that polar bears could be threatened by climate change gained traction. But he takes issue with the IPCC’s projection models for sea ice changes in the Arctic.

In 2008, he signed the controversial Manhattan Declaration on climate change, which argued that there was no conclusive evidence that carbon dioxide emissions from modern industrial activity was causing catastrophic changes in global climate.

“There was only one perspective, and that was what was provided by the IPCC,” says Taylor.

Taylor says that because he lived in the north he had direct contact with the people in the area, giving him a unique perspective on what was really happening on the ground.

“What they were describing was quite simply inconsistent with what I was hearing from local people, what I was seeing myself.”

Related posts for background and follow-up

Sea ice experts make astonishing admissions to polar bear specialists July 29, 2014

Dodgy new clarification of global polar bear population estimate (yes, another) July 5, 2014

Polar bear population numbers are for kids, says specialist Andrew Derocher [April 9, 2018]

Southern Beaufort polar bear ‘decline’ & reduced cub survival touted in 2008 was invalid, PBSG now admits March 24, 2014

Polar bears and melting ice_three facts that shouldn’t surprise you July 20, 2014

Even with Inuit lives at stake, polar bear specialists make unsupported claims April 23, 2019

Western Hudson Bay polar bears in great shape after five good sea ice seasons September 5, 2019

People go to Churchill to see polar bears in the wild and PBI controls the info they get September 18, 2019