Publication Bias And The Missing Negative Trials

The literature a reader can find is not the literature that was produced. Studies finding nothing are less likely to be written up and less likely to be published, which distorts every summary built on them.

Where the filtering happens

The first filter is the researchers themselves, who often do not submit a study that found nothing, judging it uninteresting or unlikely to be accepted.

The second is journal selection, since novel positive findings attract more attention and citation than confirmations of absence.

The third is time: null results that are eventually published tend to appear later, so a question can look settled for years before the balance shifts.

How the distortion is detected

Comparing registered trials with published ones reveals studies that were started and never reported, which is direct evidence of the gap.

Funnel plots examine whether small studies scatter symmetrically around the pooled estimate, and asymmetry suggests small unfavourable studies are missing.

These methods are indirect and can be misled by genuine differences between small and large studies, so they indicate a problem rather than measuring it precisely.

Why herbal literature is particularly exposed

Much of this research is conducted by groups with an interest in the tradition or the product, which affects both what gets studied and what gets reported.

Trial registration is less consistently practised in some of the settings where this work is done, which removes the main tool for detecting unreported studies.

Many studies also appear in journals with limited indexing, so a reader's search results depend heavily on which databases were consulted.

What is being done about it

Prospective registration of trials before enrolment is now required by many journals, and it makes unreported studies visible even when they never appear.

Some funders and regulators now require results reporting within a set period regardless of outcome, though compliance has been uneven.

Journals dedicated to publishing null results exist, and pre-registered report formats accept a study on its design before the results are known.

How a reader should adjust

The practical adjustment is to treat the visible evidence as an upper bound on how well something works rather than as a balanced picture.

Where a treatment shows only small positive studies and no large ones, the most likely explanation is that the large ones were not encouraging.

Checking a trial registry for studies on the same question is a reasonable step, and finding several completed but unreported entries is informative in itself.