Systematic Reviews and Clinical Practice Guidelines - Essay Example

Promises and pitfalls of Meta-analysis: Physicians are in need of evidence for both clinical practice and public health decision-making. Systematic reviews and clinical practice guidelines were developed for best source of evidence to counter the outburst of medical literature, the lack of time that physicians have to review all the articles, the time to critically appraise them and obtain the evidence they need for their questions. The definition of a systematic review according to the Cochrane Collaboration is “A review of clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review. Statistical methods (Meta-analysis) may or may not be used to analyze and summarize the results of the included studies” (6) [Monika Kastner, Andrea tricco] Systematic reviews (SR) are designed to provide a comprehensive summation of the available evidence for decision makers. It is seen as a way to encourage the use of research evidence in clinical decision-making. A well-conducted systematic review that uses an explicit methodology to find and synthesize all the relevant evidence, are generally considered higher caliber evidence than are individual trials in decision making for clinical practice and health policy (Meta-analysis and the Cochrane collaboration 20 years). As Green denotes “Using evidence from reliable research, to inform healthcare decisions, has the potential to ensure best practice and reduce variations in healthcare delivery.” [s.Gospalakrishnan]. The reasons behind that are due to the rigid methodology used in collecting and synthesizing all relevant studies and also for appraising each included study for risk of bias. This is done in a transparent way to allow for replication if needed. Meta-analysis is the quantitative section of a systematic review. It is the end result of combining the statistical results of multiple studies in order to get a weighted average effect of the intervention under consideration. The studies that provide more information will give more weight, the larger the sample size the more weight it will provide. Usually, the average effect across trials is reported as an overall summary point estimate and an estimate of its precision as reflected in the width of the confidence interval. [David Moher, guides for reading and interpreting…] A well-conducted systematic review and Meta-analysis can help us keep up-to-date. High-quality systematic review can define the boundaries of what is known and what is not known and helps us not to omit what has already been proven. Medical practitioners are therefore aided in determining solutions for specific clinical hurdles through analysis of inconsistencies among diverse pieces of research evidence. This is done by summarizing existing data, refining hypotheses, estimating sample size and providing a definition for future research agendas. Without them, researchers risk missing and therefore omitting promising leads in their quest for problem solution or they may incur redundancy through embarking on studies of questions that have been already answered. [Systematic reviews: synthesis of best evidence for clinical decisions, Deborah j. cook] Meta-analysis utilizes the quantitative combination of outcomes from several undertaken studies to create more precise, powerful and convincing conclusions. It also informs us of whether results of a prior analysis are applicable to specific subgroups of patients. When carefully implemented, the benefits of meta-analysis include increase in power and improvement in precision but it could also lead to serious misconceptions if some trivial aspects such as specific study designs, within study biases, variation across studies, and reporting biases are not evaluated efficiently. Cochrane book The occurrence of bias can be attributed to the publication of studies that show a significant effect of a specific treatment, publication of studies in English and citation of a study by multiple authors. This gives the studies the pre-emptive advantage of being identified raising the probability of their being included in systematic reviews therefore introducing bias. When quality of these studies is evaluated in a systematic review, the studies provide another probable source of bias. Mainly, the effects of these biases are mostly felt in small studies. This can be examined both graphically and statistically. [Systematic review sin health care investigate and dealing with publication and other biases in meta-analysis] The meta-analysis of studies with a risk of being biased may be seriously misleading. The presence of prejudice in individual studies results in errors since meta-analysis only compounds the error producing results whose credibility is misrepresented. [Cochrane book] Publication bias occurs in the difference of the readily available research from the outcomes of all other research conducted on the same. [Dickersin K: How important is publication bias? A synthesis of available data. AIDS Educ Prev 1997, 9:15-21] The outcomes of publication bias have been extensively evaluated for clinical trials to avoid exaggeration of the effectiveness of a treatment in the case of studies with statistically positive results and large effect sizes. This necessitates the inclusion of unpublished studies in the systematic reviews which often impossible to locate especially when conducted in private capacities. Variation across studies (heterogeneity) must also be considered. This can occur if differences in included studies are substantial, which could be due to clinical or methodological heterogeneity, both are important threaten to the validity of quantitative synthesis (Meta-analysis). Data combination may be inappropriate for a variety of reasons; this could be due to differences in patient eligibility criterion of the included studies, different interventions and outcomes and other methodological differences or missing information. [Guides for reading and interpreting sys r moher] The methods used for investigating heterogeneity are discussed later in this chapter. Registering systematic reviews will aid in decreasing publication biases. PROSPERO is an international registry for systematic reviews with a goal to decrease the number of unpublished SRs and hopefully decrease redundancy and increase transparency. (Reference) Methods of conducting a meta-analysis: Qualitative data synthesis: This is the systematic review methodology part which includes gathering of information to produce a summary analysis about the general characteristics of the studies included in a systematic review; that is not generated by statistical techniques which includes study characteristics such as date of publication and country of conduct, patients’ characteristics such as age range and sex of patients, characteristics of the interventions such as the dose range or the methodological quality. These qualitative data synthesis provides physicians with sufficient information about the included studies, which would aid in the appropriateness of conducting a statistical or quantitative synthesis afterward. It is important to note that Qualitative data identifies the differences between included trials i.e. heterogeneity which could threaten the validity. The PRISMA statement (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guides authors in reporting a variety of systematic reviews used when evaluating the positive and negative effects of a health care intervention. Quantitative data synthesis: Also known as meta-analysis, it occurs when at least one outcome from two or more studies included in a systematic review is statistically combined to provide an overall summary result. A variety of approaches for harmonization of statistical data in systematic reviews exist. They include Bayesian, meta-regression and frequentist approaches. We will use the most widely used method; the frequentist approach. The results selected for combination and analysis in a meta-analysis are determined by the data obtained from the included studies. Meta-analysis of binary data uses within-study variance estimates such as odds ratios, relative risk and the risk difference. Continuous data can be meta-analyzed using the mean differences and standardized mean difference. There are two models for quantitative synthesis. The first one is called the fixed effect model and the second one is the random effects model. Each model reflects a different quantitative outcome. Fixed effects model assumes that trials included in the review estimate are predetermined and therefore fixed hence the treatment effect and the observed differences across studies are simply due to chance. In the random effects model, it is assumed that included trials are only a random sample of theoretical universe of all possible trials and that their results are randomly placed around some central value therefore incorporating a between study variance. The fixed effects approach only includes a within-trial measure of variance, whereas a random effects approach includes a within and a between trial measures. The statistical properties of both models therefore differ depending on whether the studies analyzed are homogenous or heterogeneous. Research indicates that in most cases however (mainly homogenous), both approaches provide similar results since the between-study variance which differentiates the two is small or zero. Examining between-trial differences Statistical techniques may be applied to evaluate and gauge the differences between the trials included in a systematic review. Two of the most efficient statistical tests for evaluating between trial heterogeneity are the Breslow-Day test and the Cochran Q test. The former test is most applicable when examining dichotomous data, whereas the latter is used for continuous data. Due to its limited statistical power which often disable it from detection of true differences across trials, its use is limited. Subgroup analysis Subgroup analysis is the statistical analysis whose focus is on a particular subgroup of variables across all trials (such as female patients), not subgroup of trials. Instances in which the systematic reviewer, and reader, might be interested in the results on specific populations do exist. It should be relied on with caution since it is not considered to generate conclusive results. Sensitivity analysis It is the evaluation of the robustness of the results of the statistical synthesis by estimating and comparing the effects of variation in different groups of trials. The trials could be stratified according to various attributes, such as publication status, methodological quality, language of publication, and so on, to determine whether the results are robust across different subgroups of trials. A statistical analysis reports the results of all trials included in the synthesis, whereas a sensitivity analysis reports the results separately from trials of different quality. Sensitivity analysis is independent of subgroup analysis. Graphic display of the results of systematic quantitative reviews Forest plots are invaluable in submission of the results of a systematic review. An important variation to this graphic method is the graphic representation of cumulative meta-analysis where individual trials are included consecutively, according to some pre-specified order, such as year of publication or quality. Each horizontal line represents a separate systematic quantitative review. [Guides for reading and interpreting systematic reviews III, David moher] Essential principles of meta-analysis 1. Meta-analysis is a process comprised of two stages. In the first stage, a summary statistics is calculated for each study for accurate determination of the observed intervention effects. For example, the summary statistics may be a risk ratio if the data is dichotomous or difference between means if the data reflects continuity. 2. The second stage involves calculating a summary (pooled) intervention effect estimate by expressing it as a portion of the effects of the experimental intervention for each of the individual studies. A weighted average is defined as: Weighted average= sum of (estimates x weight)/sum of weights = YiWi/Wi 3. Combining the overall intervention effect estimates across studies may potentially imply that the studies are not affected by the same intervention effect since it has a defined distribution for all studies forming the basis of a random effects meta-analysis. Alternatively it is assumed that each study is estimating exactly the same quantity when a fixed-effect meta-analysis is performed. 4. By averaging the error from intervention effect, one can derive a confidence interval, which reflects the accuracy of the summary estimate allowing researchers to derive a value used to gauge the robustness of the evidence as contrasted to the null hypothesis if no intervention effect is determined present. 5. Besides providing a basis to gauge the overall effect, it is provided for in the diverse methods of meta-analysis the capability to evaluate the variation for the variety of results present as compared to the random variation and also the consistency of results among all studies is evaluated. A simple example of the meta-analysis procedure is the inverse-variance method whose application is mainly in the random effects model. This approach is applied in its most basic form in statistical programs, and it is applied in the meta-analyses of both dichotomous and continuous data. The inverse variance method is so named because the weight given to each study is inversely proportional when compared with the variance of the effect estimate. Larger studies with smaller standard errors are therefore given more weight than smaller studies, which have larger standard errors minimizing the imprecision of the pooled effect estimate. A fixed effect meta-analysis using the inverse variance method is calculated as weighted average as Generic inverse variance weighted average = Yi (1/SEi2)/ (1/SEi2) Read More