Systematic Review and Meta-Analyses - Essay Example

Systematic Review and Meta-Analyses Introduction Systematic Review is the application of scientific strategies that restrict bias to the systematic assembly, critical appraisal and synthesis of all relevant studies on a specific topic. Meta-Analysis is a systematic review that employs statistical methods to combine and summarize the results of several studies. Systematic review is a thorough, comprehensive, and explicit way of interrogating the medical literature. It typically involves several steps, including asking appropriate questions, identifying one or more databases to search, developing an explicit search strategy, selecting titles, abstracts, and manuscripts based on explicit inclusion and exclusion criteria, and abstracting data in a standardized format. Meta-analysis is a statistical approach to combine the data derived from a systematic-review. Therefore, every meta-analysis should be based on an underlying systematic review, but not every systematic review leads to a meta-analysis. Systematic Review A Systematic Review is an explicitly formulated, reproducible, and up-to-date summary of the effects of health care intervention. Systematic reviews are usually prepared by a team of at least two reviewers having proficiency in clinical domain and review methodology. In Systematic Reviews, structured method is explicitly stated at the beginning of the review. Systematic Review focuses on the objective and transparent approach for research synthesis. Many systematic reviews are based on explicit quantitative meta-analysis of available data. Some qualitative reviews adhere to the standards for gathering, analyzing and reporting evidence. Systematic Reviews help healthcare providers to evaluate and practice existing or new technologies and consider the totality of available evidence. They prove tremendously valuable in bringing together a number of separately conducted studies, sometimes with conflicting findings. (Pai et al 2004) Systematic reviews are often called overviews. Review is a general term for all attempts to synthesize the results and draw conclusions of two or more publications on a given topic. Overview is when a review strives to comprehensively identify and track down all the literature on a given topic. Overview is also called Systematic literature review. Meta-analysis is a specific statistical strategy for assembling the results of several studies into a single estimate. Most common reasons for undertaking a systematic review are to summarize existing evidence concerning a treatment, to identify gaps in current research in order to suggest areas for further investigation, and to provide a framework/background for new research activities. However, systematic reviews can also be undertaken to examine the extent to which empirical evidence supports/contradicts theoretical hypotheses, or to assist in generating new hypotheses. Cochrane Review A Cochrane Review is a Systematic Review that explores the facts/ evidence for and against the usefulness/ effectiveness and suitability/ appropriateness of treatments (medications, surgery, education, etc) in specific circumstances. Cochrane Reviews facilitate the choices that doctors, patients, policy makers and others face in the health care domain. All existing and new/ updated Cochrane Reviews are published in the Cochrane Library four times in a year. These reviews are undertaken by esteemed members of the Cochrane Collaboration which adheres to a specific methodology. The Cochrane Collaboration is an international organization helping people make informed decision about healthcare practices. The Cochrane Collaboration was founded in 1993 and it produces and disseminates systematic reviews of healthcare interventions and promotes the search for evidence in the form of clinical trials and other studies of interventions. (Higgins JPT & Green S, 2008) The Cochrane Library (which publishes Cochrane Reviews) aims to make the results of well-conducted controlled trials easily available to healthcare professionals. The Cochrane Library is considered as a key resource in evidence based medicine. Cochrane Systematic Review process involves clear question formulation, comprehensive data search, unbiased selection and extraction process, critical data appraisal, data synthesis, possible analyses of sensitivity and subgroup, and preparing a structured report. The Cochrane Library provides a collection of Cochrane Reviews in medicine and healthcare domains, a database of systematic reviews and meta-analyses which summarize and interpret the results of high-quality medical research. The Cochrane Library makes the results of well conducted and controlled trials readily available and is a key resource in evidence-based medicine. The Cochrane Library consists of the various Cochrane Databases of Systematic/ Cochrane Reviews, Abstracts of Reviews of Effects, Central Register of Controlled Trials, Methodology Reviews, Methodology Register, Health Technology Assessment, NHS Economic Evaluation. The Cochrane Reviews, CENTRAL, Methodology Reviews and Methodology Register are produced by the Cochrane Collaboration. DARE, HTA and NHS EED are compiled and maintained by the Centre for Reviews and Dissemination. Advantages and disadvantages of Systematic Reviews Their major advantage is that they provide information about the effects of some phenomenon across a wide range of settings and empirical methods. If studies give consistent results, systematic reviews provide evidence that the phenomenon is robust and transferable. If the studies give inconsistent results, sources of variation can be studied. In the case of quantitative studies, it is possible to combine data using meta-analytic techniques. This increases the likelihood of detecting real effects that smaller studies are unable to detect individually. High quality systematic reviews seek to identify all relevant published and unpublished evidence, select studies or reports for inclusion, assess the quality of each study or report, synthesize the findings from individual studies or reports in an unbiased way, interpret the findings and present a balanced and impartial summary of the findings with due consideration of any flaws in the evidence. However, Systematic reviews require considerably more effort than traditional reviews and its increased power can also be a disadvantage, since it is possible to detect small biases as well as true effects. Strengths & Weaknesses of Systematic Reviews Systematic Reviews are considered as the strongest form of medical evidence. It is a fact that Systematic reviews are cited more often than Narrative Reviews. However, it is observed that all Systematic Reviews are not equally reliable. The reporting could be improved by a deploying a set of standards and guidelines that are universally agreed upon. The same group found that 4% guidelines required updating within a year out of 100 guidelines reviewed and 11% guidelines required updating within two years. In case of rapidly changing fields of Medicine, the number of guidelines to be updated was found to be higher. Moreover, 7% Systematic Reviews required updating at the time of publication. Preparing & Maintaining Systematic Reviews There are seven steps for preparing and maintaining Systematic Reviews such as formulating a problem, locating and selecting a study, critical appraisal of the study, collecting relevant data, analyzing and presenting results, interpreting the results, and improving / updating the reviews. The information discovered based on above processes, provides valuable input to the development of safe and effective practice. Example of Systematic Review Newborn Screening is a good example of a Systematic Review. In UK, there is Bloodspot Program, in which around 700,000 babies are screened every year. In the newborn screening process, first the prior consent of the parents is obtained. Once agreed upon, the blood spots are collected from the heels of one-week old baby. These blood spots are then tested for four serious but relatively rare conditions. In this test, Phenylketonuria (PKU) and congenital hypothyroidism are tested. Apart from these two testes, National screening for Sickle Cell disorders and Cystic Fibrosis are also tested. What is Meta Analysis? A Meta-Analysis is the statistical analysis of a large collection of analyses results for the purpose of integrating the findings. (Glass, 1976, p. 3). A Meta-Analysis statistically combines the results of several studies, which address shared research hypotheses. Meta-Analysis summarizes data from individual studies to answer a specific research question. Primary objective of Meta-analysis is estimation of summary effect and estimation of differences. As research results accumulate, it increases the difficulty in understanding what they really tell us and in finding the knowledge out of the research. A method to integrate and summarize the findings from a body of research was proposed in 1976, which is called as Meta Analysis. Now- a-days, Meta Analysis is widely accepted as a method of summarizing the results of empirical studies within the behavioral, social, and health sciences. The term Meta Analysis has come to encompass all methods and techniques developed by numerous researchers. Since the pioneering work in the 1970s, literally thousands of Meta Analyses have been conducted and great improvements have been made in Meta Analysis methodology. Authors of Meta Analyses must sometimes make decisions based on their own judgment. However, meta-analysis requires that these decisions are made public so they are open to criticism from other scholars. Meta-analyses are most easily performed with the assistance of computer databases (Microsoft Access, Paradox) and statistical software (DSTAT, SAS). Some criticize that Meta-analysis simply adds together the results from different studies and calculates a summary statistic as if it is one big study. But Meta-analysis does not just add numbers from trials. Meta-analysis first looks at the results within a study, and then calculates a weighted average. Advantages of Meta-Analysis Using Meta Analysis, the population of studies can be generalized. Overall factors and size parameters in related studies can be derived and can be statistically tested. Activity between variations in study can be directed and controlled using Meta Analysis. Further, Moderators/ arbitrators can be included to explain variation in study. Using Meta Analysis, higher statistical power can be obtained to identify an effect in ‘n=1 sized study sample’. Meta Analysis has proved it’s useful in giving valuable insight into the overall effectiveness of interventions (e.g., psychotherapy, outdoor education). It is also helpful in establishing relative impact of independent variables (e.g., the effect of different types of therapy). Meta-Analysis Process The process of Meta Analysis starts with defining hypothesis. Once the hypothesis is defined, relevant literature sources are searched. Further, studies are selected based on quality criteria, and specified subject. Input data is gathered for the specific study that is undertaken. Calculation of overall effect by converting statistics to a common metric is carried out. Necessary adjustments are performed to tackle the issues like sample-size or bias. Calculation of Central Tendency (e.g., mean effect size, confidence intervals around that effect size) and variability (e.g., heterogeneity analysis) is performed. At the last, analysis of moderating variables by coding each variable in the database is carried out. In this final phase, mean differences for categorical variables as well as weighted regression for continuous variables are also analyzed. Meta-Regression Models Generally, there are three types of models in the literature on Meta-Analysis known as Simple Regression, Fixed Effects Meta-Regression, and Random Effects Meta-Regression. These methods are explained below. Simple Regression The model can be specified as where yj is effect size in study j, β0(intercept) is estimated overall effect size, is the parameter specifying different study characteristics, and is between study variation. Note that this model does not allow specification of within study variation. Fixed-Effects Meta-Regression Fixed-effects meta-regression assumes that the true effect size θ is normally distributed with  where  is the within study variance of the effect size. A fixed effects meta-regression model thus allows for within study variability, but no between study variability because all studies have expected fixed effect size θ, i.e.  where  is the variance of the effect size in study j. Fixed effects meta-regression ignores between study variations. As a result, parameter estimates are biased if between study variations cannot be ignored. Furthermore, generalizations to the population are not possible. Random Effect Meta-Regression Random effect meta-regression rests on the assumption that θ in  is a random variable following a (hyper-) distribution  where again  is the variance of the effect size in study j. Between study variance  is estimated using common estimation procedures for random effects models (restricted maximum likelihood (REML) estimators). Applications in Modern Science The basic purpose of Meta-Analysis is to provide rigorous methodology to experimental research. Meta-analysis emphasizes the shift from single studies to multiple studies. It focuses on practical importance of the effect size instead of the statistical significance of individual studies. This shift is called Meta-analytic thinking. Results from various studies are combined using different approaches. One approach frequently used in Meta-Analysis in health care research is called Inverse Variance Method. In this method, the average effect size across all studies is computed as a weighted mean, whereby the weights are equal to the inverse variance of each study’s effect estimator. Larger studies and studies with less random variation are given more weight-age as compared to smaller studies. (Egger, M & G D Smith 1997) Strengths of Meta Analysis Meta Analysis imposes a discipline on the process of summing up research findings. It represents findings in a more differentiated and sophisticated manner than conventional reviews. Meta Analysis is capable of finding relationships across studies that are obscured in other approaches. Furthermore, it protects against over-interpreting differences across studies. It can also handle a large numbers of studies (this would otherwise overwhelm traditional approaches to review). Larger Meta Analysis (i.e., those conducted with several hundred events) is likely to be more reliable and useful for healthcare professionals. In the healthcare sector, well conducted Meta Analysis of large trials using individual patient data may provide the best estimates of treatments to be followed. Weaknesses of Meta Analysis Meta Analysis is not without disadvantages and it is the subject of harsh criticism. One disadvantage of Meta Analysis is simply the amount of efforts and expertise it takes. One must admit that Meta Analysis requires a good deal of effort. Meta Analysis method does not control the sources of bias. A good Meta-Analysis of a badly designed study would results in bad statistics. This method heavily relies on published studies. This is a publication bias and it should be seriously considered while interpreting the outcomes of a Meta-Analysis. Meta Analysis causes Simpson's Paradox (two smaller studies may point in one direction but combination study may point in the opposite direction). Furthermore, mechanical aspects of Meta Analysis don’t lend themselves to capture more qualitative distinctions between studies. It is also true that comparability of studies is often in the eye of the beholder. Most Meta-Analyses include tarnished studies. Selection bias posses continual threat in negative and null finding studies that were unable to find the outcome and outcomes for which there were negative or null findings that were not reported. Analysis of study differences is fundamentally correlation. Example of Meta Analysis Patients with severe angina often require either angioplasty (PTCA) or bypass surgery (CABG). A collaborative Meta-Analysis of 3371 patients is conducted with a mean follow-up of 2.7 years. Results from eight published randomized trials are combined in the Meta Analysis. Ratio of total deaths of patients in the bypass surgery (CABG) to angioplasty (PTCA) groups was 73: 79 and Relative risk (RR) of 1.08 (95% CI 0.79-1.50). Combined endpoint of cardiac death and non-fatal myocardial infarction occurred in 169 PTCA patients and 154 CABG patients. (Lancet. 1995) Overall there was substantial similarity in outcome across the trials. Separate analyses for the 732 single-vessel and 2639 multi-vessel disease patients were largely compatible. The rates of mortality, additional intervention, and prevalent angina were slightly lower in single vessel disease. The combined evidence comparing angioplasty (PTCA) with bypass surgery (CABG) shows no difference in prognosis between these two initial revascularization strategies. However, treatments differ remarkably in the subsequent requirement for additional revascularization procedures and in the relief of angina. These results will surely influence the choice of revascularization procedure in future patients with angina. Conclusion If Systematic Reviews are rigorously conducted, they give best possible estimates of any true effect. However, we must be cautious in accepting the findings of any Systematic Review, since Systematic Reviews may be poorly conducted and not all of them are rigorous and impartial. Intervention, Patient Selection Group or Patient Search Strategy could have been done with less attention. Sometimes, Systematic Reviews are combined with Meta Analysis Study, which should not be done as they differ in intervention used or the participants. Therefore, it is important to become familiar with the steps involved in conducting Systematic Reviews and do routine appraisals of the methods used by review authors so that the bias on findings is minimized. Systematic Reviews summarize overwhelming amount of research-based healthcare information for busy healthcare providers and decision makers to be read and synthesized. They overcome the bias associated with small single trials where results may not be robust and may be subject to variation. By understanding the rational for Systematic Reviews and various steps to be followed in their conduct, clinical professionals are better empowered to comprehend and implement reliable evidence into their regular clinical practice. Meta-analysis is applicable to collections of research that are empirical rather than theoretical. Meta-analysis is applicable to collections of research that produce quantitative results rather than qualitative findings. Meta-analysis is applicable to collections of research that examine the same constructs and relationships. Meta-analysis is applicable to collections of research that have findings that can be configured in a comparable statistical form (e.g., as effect sizes, correlation coefficients, odds-ratios, etc.). Meta-analysis is applicable to collections of research that are “comparable” in the given question. Undoubtedly, Meta Analysis plays a vital role in medical research, public policy, and clinical practice sectors. It is interesting to note that its use and value will likely increase gradually, given the amount of new knowledge, the speed at which new knowledge is being created, and the availability of specialized software for performing Meta Analysis on available knowledge cubes. At the same time, it is imperative that researchers, policy-makers, and clinicians be able to critically assess the value and reliability of the conclusions that Meta Analyses provides. References “Ch 9 - Systematic Reviews”, The RM Knowledge Translation Toolkit: A Resource for Researchers, Viewed on 06 Oct 09, “An introduction to Cochrane reviews and The Cochrane Library”; The Cochrane Collaboration, Viewed on 06 Oct 09, “Systematic review”, The Citizens’ Compendium, Viewed on 06 Oct 09, “Collection of examples of service user and carer participation in systematic reviews”, User involvement in systematic reviews, Social Care Institute for Excellence, Viewed on 06 Oct 09, “Meta-Analysis”, Psychwiki, Viewed on 06 Oct 09, “Meta-Analysis”, Wikipedia, Viewed on 06 Oct 09, “Cochrane Library”, Wikipedia, Viewed on 06 Oct 09, “Examples of Meta-Analysis and Systematic Review Journal Articles”, Evidence-Based Medicine for UW Medicine Interns, UW Department of Medicine and the UW Health Sciences Libraries, Viewed on 06 Oct 09, “The Meta Analysis of Research Studies”, Glass, Evartt, & Emery (2002) A user's guide to the meta-analysis of research studies, Viewed on 08 Oct 09, “Meta Analysis Research Methodology”, by James Neill on Wilderdom, Viewed on 08 Oct 09, “Strengths and limitations of meta-analysis: larger studies may be more reliable”, by Flather MD, Farkouh ME, Pogue JM, Yusuf S., National Center for Biotechnology Information, Viewed on 08 Oct 09, “Meta-analysis of randomised trials comparing coronary angioplasty with bypass surgery”, by Pocock SJ, Henderson RA, Rickards AF, Hampton JR, King SB 3rd, Hamm CW, Puel J, Hueb W, Goy JJ, Rodriguez A., London School of Hygiene and Tropical Medicine, UK, Viewed on 08 Oct 09, “Meta-analysis: Its strengths and limitations”, Quantitative Health Sciences, Wb4, Cleveland Clinic Foundation, Viewed on 08 Oct 09, “Practical Meta-Analysis, by Lipsey, Mark W., and Wilson, David B. (2001), Applied Social Research Methods Series (Vol. 49), Thousand Oaks, CA: SAGE Publications. Read More