The scientific literature is being flooded with academic reviews of COVID-19 that are based on the same small set of primary studies, according to research published January 7 in JAMA Network Open. Researchers proved their point by analyzing studies of imaging of children with COVID-19.
The role of peer-reviewed literature during the COVID-19 pandemic has become a hotly debated topic. While some observers have welcomed the rapid dissemination of information, others have questioned the quality of studies that are being published.
In the current research letter, researchers acknowledged the important role of academic research in informing clinical decisions when treating children with COVID-19. But the explosion of medical research in the past year -- including some systematic reviews -- has seen the publication of many reviews that answer the same question using the same few primary studies.
"Replication of systematic reviews may be appropriate to verify their findings or to extend or narrow the question they are trying to answer. However, needless repetition is wasteful," wrote the authors, led by Dr. Giordano Pérez-Gaxiola, a pediatrician and director of the evidence-based medicine department at Hospital Pediátrico de Sinaloa in Culiacán, Mexico.
To illustrate the duplication problem, the authors conducted their own cross-sectional review to answer the question, "What is the spectrum and frequency of imaging findings in children with COVID-19?" They conducted their search using an artificial intelligence tool that analyzes more than 40 sources, including databases, preprint servers, and trial registries.
Giordano Pérez-Gaxiola and colleagues deemed reviews and primary studies eligible if they were published through September 1, 2020, and described imaging findings in children under the age of 18. They then plotted their findings in a matrix, helping to illustrate when reviews used the same few studies.
The team found 25 relevant systematic reviews, which cited the same 17 primary studies as sources. In some cases, multiple reviews used the exact same primary source or sources.
One primary research study, for example, appeared in 20 out of 25 systematic reviews, while another appeared in 19 of 25 reviews. At the same time, many systematic reviews included the same primary studies.
"In less than 6 months, the literature was flooded with more systematic reviews than primary studies trying to answer a very specific clinical question," the authors wrote.
Part of the problem was that only about one-quarter of the reviews had been registered on PROSPERO or another registry specifically designed to prevent the duplication of scientific research. However, Giordano Pérez-Gaxiola and colleagues also identified an additional 11 eligible studies missed by the entirety of the reviews.
"None of the systematic reviews included the totality of primary studies, which may be partly explained by the rapid rate of reporting of new studies but also by limitations of search strategies," they wrote. "This also highlights how quickly published reviews can become obsolete if they are not continuously updated."
Ironically, the authors were not immune to some of the issues described in the letter. The team noted that their matrix may be outdated by the time of publication -- a concern they noted as a limitation.
"Our analysis cannot provide clinical guidance regarding the imaging findings of children with COVID-19," they concluded. "The findings are also limited to the date of our search."
