This section introduces you to a specific methodology for reviewing the literature called systematic review.
"A systematic review is a review in which there is a comprehensive search for relevant studies on a specific topic, and those identified are then appraised and synthesised according to a pre-determined explicit method“ (Klassen, Jadad, & Moher, 1998)
Systematic reviews usually conform you six core principles and adopt specific methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review.
The six core principles:
1. A systematic review rigorously addresses clearly specified answerable questions usually derived from a policy or practice problem.
2. A broad range of stakeholders are often involved in the review. They can contribute to the development of review questions and procedures (Tranfield et al. 2003) and to that the effective dissemination of review findings to appropriate audiences (Petticrew (2001, p.100). For example, reviews for the Evidence for Policy and Practice Information Centre (EPPI Centre) on education topics frequently have short statements from head teachers, teachers, school governors and other stakeholders providing interpretations of the review findings and suggestions of how these might be implemented.
3. Extensive searches are conducted of both published and unpublished studies. The aim is to find all the studies relating to the question. Greenhalgh and Peacock (2005) have demonstrated the limitations of search strategies that solely focus on citation databases. For complex questions a systematic search should always use several methods including: searching electronic databases, hand-searching known journals, recommendations from experts and snowballing.
4. Relevance and quality criteria for the selection and inclusion of studies are designed and made explicit before the review commences. This helps ensure that reviews are impartial and balanced and prevents reviewers including only those studies that fit their particular argument. Systematic reviews are not restricted to papers published in the ‘top’ journals and ‘grey literature’ is often included. This is deemed necessary and appropriate to help overcome publication bias and the file drawer problem (where researchers file away studies with negative or neutral outcomes as they are more difficult to publish). Every individual study included in the review must meet the predetermined criteria specified for the particular review.
5. Systematic reviews summarize the findings of all the individual studies in a transparent and accessible format. Findings from individual studies are often tabulated and presented in a way so that “…other researchers, decision makers and other stakeholders can look behind an existing review, to assure themselves of its rigor and validity, reliability and verifiability of its findings and conclusions” (Pawson, 2006, p.79). Systematic reviews have processes for synthesizing multiple studies to provide results that are more than the sum of the parts. The approach to synthesis should be appropriate to the purpose of the review and the research being synthesised.
6. In relation to final review outcomes, the findings from individual studies when summarized or synthesized are often then condensed into a set of practical conclusions. Where a large body of studies provide consistent results, systematic reviews might be expected to provide reasonably clear conclusions about what it known and not known. On the other hand, if the review identifies knowledge gaps or incongruent findings, the practical conclusions are more nuanced or circumspect and raise questions for future research.
Greenhalgh, T. and Peacock, R. (2005). 'Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources', British Medical Journal, Vol. 331, No. 7524, 1064-1065.
Klassen, T.P., Jadad, A.R. and Moher, D. (1998). 'Guides for reading and interpreting systematic reviews. 1. Getting started', Arch. Pediatr. Adolesc. Med., Vol. 152, 700-704.
Pawson, R. (2006). 'Evidence-Based Policy: A Realist Perspective', London: SAGE Publications.
Petticrew, M. (2001). 'Systematic reviews from astronomy to zoology:myths and misconceptions', British Medical Journal, Vol. 322, No. 7278, 98-101.
The text on this page was created by Professor David Denyer, Professor of Organizational Change, Cranfield School of Management.