Narrative vs. Systematic Reviews in the Literature - Essay Example

Narrative vs. Systematic Reviews in the Literature ABSTRACT Reviews are one of the most popular and common ways of investigating and analyzing cause and effect of the results of any research. There are many ways of writing a review. Two ways of writing reviews, Narrative and Systematic, are compared and contrasted in this report in context of robustness, strength and weakness. INTRODUCTION A critical discussion is made in this report on the aspects of writing Narrative Reviews and Systematic Reviews. Two reviews, one each of Narrative and Systematic, are selected for comparison and contradiction. Reviews are mostly written in these two forms for pharmaceutical research which relates to the health of the people. Many test cases on animals and human beings are done in the research to investigate and deduce the accuracy, effectiveness and efficiency of every single medicine made before launching it into the market for commercial use. Writers are selected to do critical analyzes of various results of the researches. Mostly writers use systematic approach and give their view or opinion about the subject. Many others use narrative approach to describe their experiences and stories. These two different styles of writings is the subject matter of this review. FINDINGS A narrative review “Effects of chondroitin sulfate in the pathophysiology of the osteoarthritic joint” ( Martel & KwanTat & Pelletier 2009) [1] and a systematic review “The clinical effectiveness of glucosamine and chondroitin supplements in slowing or arresting progression of osteoarthritis of the knee”( Black & Clar & Henderson & MacEachern & McNamee & Quayyum 2009) [2] are selected as the case study in this report. To analyze critically, I have identified four themes in this section; Significance of Reviews, Writing Narrative Reviews, Writing Systematic Reviews and Comparing Narrative vs. Systematic Reviews. Significance of Reviews The core objective of writing a review is to “imply careful examination of something, formulation of a judgment, and statement of the judgment, usually in written form.” (Reviews 2011) [3] Influence Clinical Decision Making The first importance of a review is that it influences and assists the clinical decision making. Narrative and Systematic reviews are the biomedical literature which doctors read all around the world while taking clinical decisions. Practitioners keep themselves updated with the large number of evidences reported in the reviews. This helps them to understand the differences among studies on the same question. All doctors and practitioners incorporate research evidence into clinical decision making. Reviews are “used increasingly to inform medical decision making, plan future research agendas, and establish clinical policy.”(Ann Intern 1997) [4] Gives Panoramic View The second importance of a review is that it gives an unobstructed and wide view of an extensive area on all directions through its different types. Qualitative systematic review gives only the results of primary studies, Quantitative systematic reviews are often known as meta-analysis and uses statistical methods to deduce results of two or more studies. Narrative reviews lack explicit descriptions of systematic methods. Integrative publications, practice guidelines, economic evaluations and analysis of clinical decision are all different varieties of reviews that are written by innumerable people on one specific subject from different angles, aspects, culture and use. “Cochrane Reviews are designed to facilitate the choices that practitioners, consumers, policy-makers and others face in health care. No other organisation matches the volume, scope and range of healthcare topics addressed by Cochrane Reviews.”(Cochrane 2011) [5] Brings Together Different Thoughts The third importance of a review is that it brings together one unanimous and common conclusion for the benefits of the people. Writing of each person is different based on his knowledge, qualifications, intellect and environment. When the network of such people get together, they find one conclusion for all similar researches derived through various thoughts and knowledge processes of their own. There are number of databases now specially designed for the purpose of networking, collaboration and co-operation among all the related people to attain this goal. “When considering usage across all web platforms (including Wiley InterScience, cochrane.org, La Bibliotheca Cochrane Plus, Bireme, OVID and EBSCO), it is estimated that in 2009, there was a search on T he Cochrane Library every 1 second, an abstract viewed every 2 seconds and a full text review downloaded every 3 seconds .”(Cochrane 2011) [5] Makes Bias Visible The fourth importance of a review is that it makes visible a writer’s tendency or inclination, especially one that prevents unprejudiced consideration of a question. The bias comes in to limelight when reviews of other writers on the same topic come out. “The tendency for authors to submit, and of journals to accept, manuscripts for publication based on the direction or strength of the study findings has been termed publication bias.”(Hopewell 2009) [6] Writing Narrative Reviews Narrative reviews are one of the most primitive ways of critical analyzes. Authors of narrative reviews discuss and summarize the literature on a particular topic. It is an account or narration of the author on the topic, with references to different experiences and stories and with out any scientific and mathematical technique. The results of individual studies are not combined and statistically analyzed. “Narrative is an interpretive approach in the social sciences and involves using storytelling methodology. The story becomes an object of study, focusing on how individuals or groups make sense of events and actions in their lives.”(Mitchell & Egudo 2003) [7] In narrative reviews, the focus is on how individuals or groups make sense of events and actions in their lives through examining the story, and the linguistic and structural properties (Riessman1993). [8] Practitioners, doctors and people-involved in health care listen stories and discourses of their patients about what, why, where and when they experienced a problem and how it was treated previously. They then tell these narratives or write them in reviews to create hopes, favorable attitude and positive behavior in their patients and colleagues. “In psychoanalysis through dealing with clients, therapists listen to individuals’ narrations of their lives and transform these narrations into alternative narratives that are more adaptive, more coherent and functional (Polkinghorne 1988). [9] Writing Systematic Reviews The improvement in technologies and knowledge base brought new approaches for the scientific investigations and methodologies which were far more rapid, accurate, economical, efficient and effective. This also brought change in the way of writing reviews. More and more scientific analysis, statistical analysis through graph, tables and pictorials brought new face to the narrative style of writing. This new face is today known as Systematic Reviews. As the systematic reviews use scientific and mathematical methods, they are more precise. Each review is based on one specific question and all the analysis of different nature revolves around it. It is a synthesis of research on a particular subject mostly related to medicine. “Since the early 1990s the ‘science’ of evidence review and synthesis has had a growth spurt and has developed rapidly. Today there are a number of well regarded handbooks offering guidance on the conduct of what have become known as systematic reviews of research evidence. These are largely in the field of healthcare, but increasingly in the social sciences more generally (CRD, 2001; [10] Alderson et al., 2004; [11] Petticrew and Roberts, 2005; [12]) Comparing Narrative vs. Systematic Reviews The comparison between narrative and systematic reviews is done in context of the two reviews taken as the case study. The essence of comparison and contradiction amongst these two types of reviews are; Research Question, Sources of Information, Approach of Reviewing, Authors, Audience, Magnitude of Quality, Relevancy of Evidence, Quantum of Findings, Consequences and Applicability. Research Question The question of narrative review is quite broad; it emphasizes the drug “Chondroitin Sulphate” and does not specify what number of effects it causes or which specific effect it is going to cure most. In fact, the research question gives the impression that it is going to tell us about all the possible effects of this medicine when used. The question of systematic review is very specific; it emphasizes “progression of osteoarthritis of the knee” and that too is further narrowed by the word “slowing or arresting”, when the supplements are used. Sources of Information The source of information in narrative review is limited and incomplete which create doubts about its authenticity and verifiability. As an example the sentence “For decades, it was believed that osteoarthritis consisted” talks about belief but do not say who, how and when it was believed. It just talks about a general assumption with no reference or evidence. Many such narrations are found in the review report. Most of the information given appears to be the personal experience of authors or their friends or some other people from whom they have heard from. The figures and tests referred do not give any convincing evidence about the results except that a story is told about what has happened and when it was done. How, when, why are not discussed at all. The constraints and conflicts during investigation are not accounted for. The sources of information in systematic review are quite elaborate, specific and complete. It creates no doubt as to its authenticity and verifiability. As an example the definition and criteria given for osteoarthritis is taken from the “American College of Rheumatology definition of OA” Each statement is supported by numerous reports and analysis. There are no stories or experiences; they are exact real life facts which are tabulated and analyzed to reach a result. Approach of Reviewing The approach used in narrative review is theoretical and individual based. For example, the role of chondroitin sulfate is given in the narrative review as major. “The high content of chondroitin sulfate in the aggrecan plays a major role in allowing cartilage to resist tensile stresses during various loading conditions by providing the tissue with resistance and elasticity” [1] It does not give any evidence to prove the role as “major, such as tabulated data, experiments done over the past several years on human beings or any such researches. There is no substantial evidence to prove that the role of chondroitin sulfate is truly major in real world as well. The approach used in systematic review is evidence based. Each and every statement is based on research studies conducted for long time with critical monitoring of data collection. There are results of number of studies, their analysis and their statistics which helps to conclude “the clinical effectiveness and cost-effectiveness of glucosamine sulphate or hydrochloride and chondroitin sulphate in modifying the progression of OA of the knee.” [2] Authors The most important factor that determines the value of the review is its author. The reviews are personified by the quality and creditability of its author. It is therefore very important to know who the authors are and what are their qualifications, education, experiences and role in the review. In the case study, there are only two to three authors of narrative review. There is no mention of their values such as qualifications, education and experiences. The only information given is on where to approach them e.g. “Osteoarthritis Research Unit, University of Montreal Hospital Research Centre (CRCHUM), Notre-DameHospital, 1560 Sherbrooke St. East, Montreal, Quebec, Canada H2L4M1.Tel:1-514-890-000; Fax:1-514-412-7582. E-mail address: jm@martelpelletier.ca (J. Martel-Pelletier) [1]. In the systematic review there are eight authors; name of each one is given with his designation in the Acknowledgement section with their major contributions. The role of authors is of prime value as they are the trustee and any fabrication, malafide intention and biased inclination can cause great loss to the people and the cause for which the reviews are done. “Misconduct can seriously distort the scientific research record, and several articles have indicated that misconduct is widespread. Fanelli et al. found that 2% of scientists surveyed admitted to having fabricated, falsified or modified data or results at least once, and up to 34% admitted other questionable research practices; in surveys asking about the behaviour of colleagues, rates were 14% for falsification, and up to 72% for other questionable research practices. [13] Audience Audience is the people who are going to use the reviews. The author must have the knowledge of the audience for which the review is written and he must plan the contents accordingly. The narrative review under discussion has only five pages of information and it appears to address the audience of students. Mostly the narrative reviews are written for the education sector. The systematic review under discussion has 162 pages and it is a highly professional review with complete table contents, evidence of reports, analysis, tabulated data, and inferences. The audience seems to be the global health-care market which can be doctors, practitioners, students, consumers, health-care workers or government policy makers. Magnitude of quality There are several occasions when quality of the reviews are not found up to the mark as is evident from this incidence. “Schlosser and Goetze (1992) compared four reviews that were focused on the effectiveness of treatments to reduce self-injurious behavior in individuals with developmental disabilities. The reviews varied widely not only in terms of included studies (by comparing only the overlapping years) but also in terms of conclusions about the effectiveness of various treatments.”[14] The magnitude of quality is the attributes of a review as per a given standard. These attributes include Protocol, Question, Sources, Scope, Selection Principles, and Data Extraction. Protocol is a set of defined rules and procedures to be followed while writing a review. Question is the precise description of the topic that is to be addressed. Sources include “general-purpose databases, search engines, meta-search engines, journals, personal or published bibliographies, trials registers, conference proceedings, book chapters, books, papers, reports, technical notes or other documents produced and published by governmental agencies, academic institutions and other groups that are not distributed or indexed by commercial publishers main research.” (Ralph 2007) [15] Scope is the extent of the review in terms of geographical conditions, time available for the purpose of search and the language to be used for the review. Selection Principles are the criteria set for the inclusion or exclusion of a study in the review. “Along these lines, a log of rejected studies along with reasons for their rejections should be made available either in an appendix or upon request from the authors (Auperin et al., 1997)” [16] Data Extraction refers to the details of how the data was extracted for various studies referred in to the review. A complete details of the methods used and procedures followed are given in a separate section of each study. If we compare these traits of quality we find that in narrative research, no protocol is used whereas in systematic reviews protocols defined by the HTA organization is used. Question of narrative research is broad while that of systematic research is precise. Sources of narrative research are limited and incomplete and that of systematic review are evidence based, exhaustive and elaborated. Scope of narrative research is global education, with limited searches in English and that of systematic research is global health-care, with wide range of searches in English and other foreign languages. Selection Principles for narrative research is missing and where as systematic review gives a complete list of excluded reviews with the reason for exclusion in Appendix 2 and including criteria in Appendix 5 and 6. Data Extraction in narrative research is missing whereas in systematic research Appendix 1 lists a complete strategy of Search with details of search terms and no of hits. Relevancy of Evidence In narrative review, no supporting evidence in the form of meta-analysis is given whereas in Systematic reviews Appendix 3 and 4 give meta-analysis of total number of studies included with their results. Quantum of Findings In narrative research there is no way to measure the quantum of findings either qualitatively or quantitatively whereas in systematic research a complete chapter (six) is written with discussion and recommendations. Consequences and Applicability The consequence of narrative research is only a suggestion to experiment and try the medicine. “All these findings provide interesting and insightful information about the mechanisms by which chondroitin sulfate and its combination with glucosamine sulfate may exert the in vivo beneficial effects on the osteoarthritis disease process as shown in some clinical trials” [1] The consequence of systematic research is a set of specific and precise recommendations as given in chapter six that would help the audience to take the clinical decision. “For chondroitin, biologically plausible mechanisms have been reported, but the clinical effectiveness evidence was less convincing with greater heterogeneity in conclusions. Trials of chondroitin, however, have tended to be of poorer quality, so uncertainties about the clinical effects of chondroitin remain.” [2] ROBUSTNESS, STRENGTH AND WEAKNESS I have prepared Table 1 to compare robustness, strength and weakness comparing the narrative and systematic research based on the criteria and discussions done earlier. Based on Table 1, pie chart is made for comparing Robustness, Strength and Weakness in percentages. Table 1 Comparison or Robustness (No=0, Less=1, More=2, Most=3) Criteria Narrative Review Systematic Review Rank Reasons Rank Reasons Topic Question No Very broad Most Precise and specific Source of Information Less Limited and incomplete Most Evidence based with Meta-Analysis of every single study Approach of Reviewing Less Theoretical and narrative Most Practical and real world facts Authors Less No background given More Names of all authors with their contributions given Audience More Students Most Doctors, practitioners, students, consumers, health-care workers or government policy makers Quality Protocol No No protocol is used Most HTA Protocol used Selection Principle No No evidence given Most Criteria for including and excluding reviews are given Data Extraction No No evidence given Most A complete list of search strategy with terms used and no. of hits. Relevancy of Evidence No No evidence given Most Meta-Analysis given in Appendix 3 and 4 Quantum of Findings Less Only suggestions based on narration Most Precise and explanatory in Chapter 6 Consequences &Applicability. Less Suggested to try out at your own risk Most Precise recommendation for clinical decision making Total Rank 7/33 32/33 CONCLUSIONS In comparing two cases, one each of Narrative and Systematic Review, it is concluded that narrative review is only 2% robust, 96% weak and has 18% strength when compared to Systematic Review which is 98% robust, 4% weak and has 82% strength. Systematic Reviews are far more reliable and conclusive for use. REFERENCES [1] Martel, KwanTat, Pelletier 2009, ‘Effects of chondroitin sulfate in the pathophysiology of the osteoarthritic joint: a narrative review’, Osteoarthritis and Cartilage, 2010 June18 Suppl 1:S7-11. Epub 2010 Apr 27, viewed 20 May 2011, http://www.ncbi.nlm.nih.gov/pubmed/20399897 [2] Black C, Clar C, Henderson R, MacEachern C, McNamee P, Quayyum Z, et al. ‘The clinical effectiveness of glucosamine and chondroitin supplements in slowing or arresting progression of osteoarthritis of the knee: a systematic review and economic evaluation.’ Health Technology Assessment, 2009; 13(52), viewed , 20 May 2011, http://www.hta.ac.uk/projectdev/1717.asp [3] Reviews. Dictionary.com Unabridged. Random House, Inc.. Viewed 19 May. 2011. . [4] Ann Intern Med. 1997;126:376-380. ‘Annals of Internal Medicine’, is published twice monthly and copyrighted © 1997 by the American College of Physicians. [5] Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication ‘bias in clinical trials due to statistical significance or direction of trial results.’ Cochrane Database of Systematic Reviews 2009, Issue 1. Art. No.: MR000006. DOI: 10.1002/14651858.MR000006.pub3. [6] Cochrane Reviews, ”Cochrane Collaboration”, viewed May 20, 2011, http://www.cochrane.org/cochrane-reviews [7] Mitchell M, Egudo M. ‘A Review of Narrative Methodology’. Public Release by Defence Science and Technology Organisation, 2003; DSTO-GD-0385, viewed 20 May 2011, http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.130.1381.pdf [8] Riessman, C.K. (1993), Narrative analysis, Newbury Park, CA, Sage. [9] Polkinghorne, D.E, (1998), Narrative knowing and the human sciences, Albany, NY, State of New York University Press. [10] CRD (2001) Undertaking systematic reviews of research on effectiveness: CRD’s guidance for those carrying out or commissioning reviews. CRD Report No. (2nd edn). York: University of York Centre for Reviews and Dissemination. www.york.ac.uk/inst/crd/report4.htm [11] Alderson, P., Green, S. and Higgins, J.P.T. (eds) (2004) Cochrane reviewers’ Handbook 4.2.2. In: The Cochrane Library, Issue 1. Chichester, UK: John Wiley & Sons. http://www.cochrane.org/resources/handbook/hbook.htm [12] Petticrew, M. and Roberts, H. (2005) Systematic reviews in the social sciences: a practical guide. Oxford: Blackwell [13] Fanelli D. How many scientists fabricate and falsify research? A systematic review And meta analysis of survey data. PLoSONE 2009;4:e5738, viewed 20-May-2011, http://www.thecochranelibrary.com/details/editorial/886689/The-role-of-Cochrane-Review-authors-in-exposing-research-and-publication-iscond.html#_5_ [14] Schlosser, R. W. (2003). Synthesizing efficacy research in AAC. In R. W. Schlosser, The efficacy of augmentative and alternative communication: Towards evidence-based practice (pp. 230-258). San Diego, CA: Academic Press [15] Ralf W. Schlosser. Appraising the Quality of Systematic Reviews. FOCUS Techincal Brief No. 17 2007, viewed 20-May-2011, http://www.ncddr.org/kt/products/focus/focus17/ [16] Auperin, A., Pignon, J.-P., & Poynard, T. (1997). Review article: Critical review of meta-analyses of randomized clinical trials in hepatogastroenterology. Aliment Pharmacological Therapy, 11, 215-225. viewed 20-May-2011 http://www.ncddr.org/kt/products/focus/focus17/ . Read More