Research
What a systematic review adds
It is not a longer opinion piece with more citations. A systematic review commits to its question before it looks, searches for everything including the studies nobody published, and rates what it finds.
The difference from a review article
Two things get called reviews and they are not the same object.
A narrative review is an expert writing about a subject, citing the work they consider relevant. It can be excellent. Which studies appear in it depends on which studies the author knew, chose and liked.
A systematic review starts from a question, writes down how it will search and what it will accept, and then goes looking. What it finds is what it reports, including results that cut against the answer anyone expected.
The second one is a method. That is the whole of the difference, and it is the reason the two carry different weight.
The commitment happens first
The Cochrane Handbook, which is the standard reference for how these are done, states the aim in its opening chapter.
Systematic reviews aim to minimize bias through the use of pre-specified research questions and methods that are documented in protocols, and by basing their findings on reliable research.
The order in that sentence is load-bearing. The question and the methods are fixed before the results are known, and they are written down where someone else can check them.
The handbook puts the same idea a second way. The review question should determine the methods used in the review, and not the other way round.
It is also honest about the strain in this. A review is usually retrospective, so some knowledge of the evidence is often unavoidable. That is one reason the handbook says a review team should include methodologists as well as subject experts.
Who is supposed to do one
The handbook is specific about the team, and the requirement is worth quoting because it is often ignored by things that call themselves reviews.
A systematic review should be conducted by a team that includes domain expertise and methodological expertise, and who are free of potential conflicts of interest.
It also says the people who might make, or be affected by, decisions about the intervention should be involved in important decisions about the review.
A review of a product written by the people selling it can be careful and can still fail that first condition on its face.
The search is the part that does the work
What separates a systematic review from a well-read essay is the effort to find studies the author would rather not have found.
That matters because the published literature is not a fair sample of the research that was done, and the size of the tilt has been measured.
A methodology review pooled 425 reports covering 307,028 abstracts presented at scientific meetings. Only 37.3 percent were ever published in full, with a confidence interval of 35.3 to 39.3 percent.
The missing ones are not missing at random. Abstracts reporting a positive result were more likely to reach full publication, with a risk ratio of 1.31.
So anyone who reads only what is easy to find is reading a version of the evidence that has been filtered toward good news. A systematic search is the attempt to unfilter it.
A meta-analysis is a separate, optional step
The two terms get used interchangeably and they describe different things.
A systematic review is the whole process: the question, the search, the selection, the appraisal and the synthesis.
A meta-analysis is one way of doing the last part, combining numerical results from several studies into a single estimate. Many systematic reviews contain one. Many correctly do not.
A review declining to pool its studies is often making a judgment rather than dodging one. Studies that measured different things in different populations should not be averaged, and combining them would manufacture a precise-looking number out of incompatible parts.
Reviews rate studies, they do not just count them
The other habit that distinguishes the method is appraisal. Each included study is assessed for how much its result can be relied on, and that assessment is reported.
This is why a review can conclude that ten studies exist and the evidence remains weak. Ten small, uncontrolled, short studies do not add up to one good one.
It also cuts the other way. A review can find few studies and still reach a firm conclusion, if those studies were large and well conducted.
The count is the least informative number in a systematic review, and it is the one most often quoted from it.
How a good one identifies itself
There is a published reporting standard, and it is the fastest way to tell a real review from something wearing the name.
The PRISMA 2020 statement is a guideline for reporting systematic reviews, updated to reflect advances in methods to identify, select, appraise and synthesize studies. It sets out a 27-item checklist, an accompanying checklist for abstracts, and flow diagrams for original and updated reviews.
A review following it tells you its eligibility criteria, and the databases it searched and when. It also gives the number of records screened, the number excluded and why, and how it judged the reliability of what it kept.
That flow diagram is the single most useful thing on the page. It shows the funnel from everything found to everything used, and a review unwilling to show it is asking to be trusted on the part that matters.
Reviews conducted to this standard also disclose their authors' competing interests in detail, down to editorial roles and payments from named companies. The disclosure is part of the output, not an afterthought.
What an empty review proves
A systematic review can search thoroughly and find nothing eligible. That result is not a failure of the review.
It is one of the strongest findings available in this market, because it is an absence with a method attached. Somebody specified what would have counted, looked in named places on a named date, and reported what was there.
That is a different object from a sentence asserting no studies exist. The first can be checked and repeated. The second is a claim about the writer's knowledge.
The Cochrane Handbook describes this use directly, noting that a review will indicate where specific gaps in knowledge exist or evidence is lacking, and that one should typically be done before new primary research begins.
What it still cannot fix
A systematic review is a method for handling a literature, not a machine for producing certainty.
It cannot improve the studies it found. A careful review of six weak trials reports weak evidence, and the care shows in how plainly it says so.
It cannot recover research that was never written down anywhere, and it can only estimate how much is missing.
And it does not settle a claim on its own. Federal advertising guidance points at the same idea from the regulatory side, weighing the totality of the evidence and warning that a stronger body of contrary evidence can make even a carefully qualified claim deceptive.
Reading one in about two minutes
Find the question. It should name the population, the intervention, the comparison and the outcome, and it should be narrow.
Find the search date. A review is a photograph of a literature on a particular day, and an old one may have been overtaken.
Find the flow diagram. How many records were screened, how many studies were included, and what the exclusions were for.
Find the appraisal. What the review says about the quality of what it included, not just how much of it there was.
Then read the conclusion against those four things. A confident conclusion resting on a handful of small studies is a mismatch you can see without any expertise at all.
Key takeaways
- A systematic review fixes its question and methods in a protocol before searching; a narrative review does not.
- The Cochrane Handbook says review teams should combine domain and methodological expertise and be free of conflicts of interest.
- The search exists to counteract publication bias: only 37.3 percent of 307,028 conference abstracts were ever published in full, and positive ones reached publication more often.
- A meta-analysis is an optional synthesis step, not a synonym for a systematic review.
- Reviews appraise the reliability of each study, so ten weak studies can still amount to weak evidence.
- PRISMA 2020 sets a 27-item reporting checklist plus flow diagrams, and the flow diagram is the fastest thing to check.
- A review that finds no eligible studies produces an absence with a stated method behind it, which is far stronger than an unsupported claim that none exist.
Frequently asked questions
How is a systematic review different from a review article?
A narrative review is an expert writing about a subject and citing what they consider relevant, so its contents depend on what the author knew and chose. A systematic review fixes its question and its methods in a protocol before searching, then reports what the search returned. The Cochrane Handbook describes the aim as minimizing bias through pre-specified research questions and methods documented in protocols, and basing findings on reliable research.
Is a systematic review the same as a meta-analysis?
No. A systematic review is the whole process of question, search, selection, appraisal and synthesis. A meta-analysis is one optional way of doing the synthesis, combining numerical results into a single estimate. Many systematic reviews contain no meta-analysis, and declining to pool incompatible studies is usually a judgment rather than an omission.
Why does the search matter so much?
Because the easily found literature is tilted. Pooling 425 reports covering 307,028 conference abstracts, a methodology review found only 37.3 percent were ever published in full. Abstracts with positive results reached publication more often, with a risk ratio of 1.31. Reading only what surfaces easily means reading a version of the evidence already filtered toward favorable answers.
What is PRISMA?
A published standard for reporting systematic reviews. The PRISMA 2020 statement presents a 27-item checklist, a separate checklist for abstracts, and flow diagrams for original and updated reviews. A review following it states its eligibility criteria, the databases searched and when, how many records were screened and excluded and why, and how the reliability of the included studies was assessed.
Is a review that found no studies a wasted effort?
No, it is one of the more useful results in this market. An empty review is an absence with a method attached: someone specified what would have counted, searched named sources on a named date, and reported the outcome. That can be checked and repeated, unlike a bare assertion that no studies exist. The Cochrane Handbook notes that reviews indicate where gaps in knowledge or evidence exist.
Does a systematic review settle the question?
Not by itself. It cannot improve the studies it found, so a careful review of weak trials reports weak evidence. It cannot recover research that was never recorded anywhere. A body of evidence also has to be weighed as a whole. Federal advertising guidance warns that where a stronger body of contrary evidence exists, even a carefully qualified claim is likely to be deceptive.
Sources
Each document below is named as it names itself, with the date printed on that document rather than the day it was read.
- Chapter 1: Starting a review, in the Cochrane Handbook for Systematic Reviews of Interventions, version 6.5 — Cochrane, August 2021
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviews — BMJ, volume 372, article n71 (PubMed identifier 33782057), March 2021
- Full publication of results initially presented in abstracts — Cochrane Database of Systematic Reviews, issue 11, article MR000005 (PubMed identifier 30480762), November 2018
- Health Products Compliance Guidance — U.S. Federal Trade Commission, December 2022