Critical Evaluation Tool for Appraising Systematic Reviewsof Systematic review - Assignment Example

EVALUATION TOOL FOR APPRAISING SYSTEMATIC REVIEWS Adapted from the Critical Skills Appraisal Programme (CASP) of Review: Interventions for Learning Disabled Sex Offenders Author(s): Lorraine Ashman and Lorna Duggan Source: Ashman L, Duggan L. Interventions for learning disabled sex offenders. Campbell Systematic Reviews 2004.3 Question 1 Did the review ask a clearly focused issue? Consider if the question is ‘focused’ in terms of: - the population studied the intervention given the outcomes considered YES CAN’T TELL NO Comments The review clearly focuses on the effects of intervention for learning disabled sex offenders. No doubt, the review will help improve policy and services with regard to interventions for disabled sex offenders. As it is difficult for sex offenders with learning disabilities to respond positively to therapies and medication it becomes a great challenge to design successful treatment approaches for them. However, in the outcomes of the study, the authors could not find any ‘randomised controlled trial evidence’ that would be adopted in the treatment of learning disabled sex offenders (p. 3). Question 2 Did the review include the right type of study? Consider if the included studies: Address the review’s question have an appropriate study design YES CAN’T TELL NO Comments The primary purpose of the review was “to determine the efficacy of interventions with learning disabled sex offenders” (p.4). The reviewers have successfully employed the right appropriate study design; the study design comprised of reviewing such electronic materials as the ‘EMBASE, PsycINFO, Medline, Cinahl, Cochrane Library, SPECTR, National Clearinghouse on Child Abuse and Neglect Information and National Criminal Justice Reference Service and Biological Abstracts’ and the selection criteria was based upon all randomised controlled trials by which the reviewers could compare one intervention for learning disabled sex offenders to another so that they could come to certain conclusions. Similarly, the data obtained from such electronic resources were independently extracted and analysed. As such, one can understand that the selected study design is capable of answering the research question regarding the effectiveness of intervention programs for the disabled sex offenders. Question 3 Did the reviewers try to identify all relevant studies? Consider * which bibliographic databases were used? * If there was follow-up from reference lists? * If there was personal contacts with experts? * If the reviewers searched for unpublished studies? * If the reviewers searched for non-English language studies? YES CAN’T TELL NO Comments The reviewers, in fact, have made some remarkable attempts to identify all relevant studies on the chosen issue. The bibliographic databases were consisted of electronic resources, resources from relevant articles, unpublished data and unpublished trials, and interviews with various authors. The unpublished data and unpublished trials were obtained from contacts with Schering HC and Pharmacia Ltd, two leading pharmaceutical companies that market ant libidinal medication in the U.K. The reviewers also made special efforts to follow-up from reference lists by searching referenced articles for further relevant trials. The reviewers sought relevant information from the authors of various studies whenever there was either a need to clarify data or to get additional information on studies. Similarly, certain authors and studies were excluded from the review as they did not help answer the research question. To conclude, it can be stated that the reviewers have really tried hard to gather all possible relevant studies on the issue. Question 4 Did the reviewers assess the quality of the included studies? Consider: * if a clear, pre-determined strategy was used to determine which studies were included. Look for: a scoring system more than one assessor. YES CAN’T TELL NO Comments The reviewers have done a remarkable effort to access the quality of the included studies by excluding irrelevant materials and applying many rating scales to measure the outcomes in mental health and criminological trials (p.7). The reviewers agree with the fact that the measuring instruments should have the properties of reliability and validity. In order to explore the desired effect, the reviewers adopted certain outcome measures. The primary outcome measures used were; recidivism, people lost to follow up and psychometric scores. The reviewers selected material also by regarding other outcomes like, death, other non-sexual offence, adverse effects and level of security of placement. For this study, the reviewers gave prominence to the materials, either in the form of a self report; or completed by an independent rater or relative (p.7). They gave priority to the electronic materials, materials from relevant articles, contacting with pharmaceutical companies marketing antilibidinal medication in the United Kingdom and by contacting with authors of relevant studies. Thus, it makes clear that the reviewers have resorted to various ways and had a clear pre-determined strategy to assess the quality of the included studies. Question 5 If the results of the studies have been combined, was it reasonable to do so? Consider whether: the results of each study are clearly displayed the results were similar from study to study (look for tests of heterogeneity) the reasons for any variations in results are discussed YES CAN’T TELL NO yy Comments While evaluating the review, one can understand that the reviewers have attempted a systematic study by compiling the results of all the data they used for the study. It is quite evident while through the types of materials the reviewers taken for this study. The reviewers mainly depended on two American databases, reference lists, contacted authors of studies and contacted pharmaceutical companies. The reviewers thoroughly scrutinized each article with the help of different theories in order to reach the main result. An overview of the review reveals that the reviewers have displayed the results of each study to help support the topic. Regarding this fact, one can observe that it is quite reasonable to combine different articles. If the reviewer combines different articles, it will help the reviewer to reach in an authentic point of view with regard to the concerned topic. Question 6 How the results are presented and what is the main result? Consider how the results are expressed (e.g.odds ratio, relative risk, etc.) how large this size of result is and how meaningful it is how you would sum up the bottom-line result of the review in one sentence Comments The result of the review has been presented very systematically with the support of the various materials adopted for the study. The two reviewers adopted various methods to gather materials for the study. After gathering the data, the two reviewers independently inspected all reports of identified studies. Then the reviewers focused on the assessment of methodological quality, allocating trials to three quality categories, A - adequate concealment, B - concealment unclear and C -inadequate concealment, as described in the Cochrane Collaboration Handbook (p.10). Addressing publication bias with the help of a funnel graph is the next stage of the evaluation. Later, the reviewers independently extract the data. Before reaching the final result, the following steps, data synthesis, intention to treat analysis, binary data, continuous data and test of heterogeneity are also included in the study. The study leads the reviewers to the final result, that is, “no randomised results controlled trials that fulfilled the inclusion criteria.” Question 7 How precise are the results? Consider: If a confidence interval was reported. Would your decision about whether or not to use this intervention be the same at the upper confidence limit as at the lower confidence limit? If a p-value is reported where confidence intervals are unavailable Comments The results of the review are precise enough for anyone to understand easily. For the review, the reviewers have used a confidence interval to help support the study. As the reviewers could not find out any usable data, they resort to binary data in order to calculate the relative risk (RR) statistic with 95% confidence interval and use a random model. Regarding the review, it is clear that using intervention is equal to upper confidence limit or lower confidence limit. As the reviewers have adopted more than one mean, one has to consider the fact that the confidence interval generates a lower and upper limit for the mean. It also lets the readers to calculate how much uncertainty there is in our estimate of the true mean. One should also regard that if the interval is narrow, the estimate will also be precise. The reviewers have used p-value where confidence intervals are unavailable (p.11). Question 8 Will the results help locally? Consider whether: the population sample covered by the review could be different from your population in ways that would produce different results your local settings differ much from that of the review you can provide the same intervention in your setting YES CAN’T TELL NO Comments The results of the review can be used locally because of the choice of the participants and type of the study. The study adopted was relevant randomised controlled trials and for this study, “Males or females with learning disability (defined as IQ Read More

Consider: * if a clear, pre-determined strategy was used to determine which studies were included. Look for: a scoring system more than one assessor. YES CAN’T TELL NO Comments The reviewers have done a remarkable effort to access the quality of the included studies by excluding irrelevant materials and applying many rating scales to measure the outcomes in mental health and criminological trials (p.7). The reviewers agree with the fact that the measuring instruments should have the properties of reliability and validity.

In order to explore the desired effect, the reviewers adopted certain outcome measures. The primary outcome measures used were; recidivism, people lost to follow up and psychometric scores. The reviewers selected material also by regarding other outcomes like, death, other non-sexual offence, adverse effects and level of security of placement. For this study, the reviewers gave prominence to the materials, either in the form of a self report; or completed by an independent rater or relative (p.7). They gave priority to the electronic materials, materials from relevant articles, contacting with pharmaceutical companies marketing antilibidinal medication in the United Kingdom and by contacting with authors of relevant studies.

Thus, it makes clear that the reviewers have resorted to various ways and had a clear pre-determined strategy to assess the quality of the included studies. Question 5 If the results of the studies have been combined, was it reasonable to do so? Consider whether: the results of each study are clearly displayed the results were similar from study to study (look for tests of heterogeneity) the reasons for any variations in results are discussed YES CAN’T TELL NO yy Comments While evaluating the review, one can understand that the reviewers have attempted a systematic study by compiling the results of all the data they used for the study.

It is quite evident while through the types of materials the reviewers taken for this study. The reviewers mainly depended on two American databases, reference lists, contacted authors of studies and contacted pharmaceutical companies. The reviewers thoroughly scrutinized each article with the help of different theories in order to reach the main result. An overview of the review reveals that the reviewers have displayed the results of each study to help support the topic. Regarding this fact, one can observe that it is quite reasonable to combine different articles.

If the reviewer combines different articles, it will help the reviewer to reach in an authentic point of view with regard to the concerned topic. Question 6 How the results are presented and what is the main result? Consider how the results are expressed (e.g.odds ratio, relative risk, etc.) how large this size of result is and how meaningful it is how you would sum up the bottom-line result of the review in one sentence Comments The result of the review has been presented very systematically with the support of the various materials adopted for the study.

The two reviewers adopted various methods to gather materials for the study. After gathering the data, the two reviewers independently inspected all reports of identified studies. Then the reviewers focused on the assessment of methodological quality, allocating trials to three quality categories, A - adequate concealment, B - concealment unclear and C -inadequate concealment, as described in the Cochrane Collaboration Handbook (p.10). Addressing publication bias with the help of a funnel graph is the next stage of the evaluation.

Later, the reviewers independently extract the data. Before reaching the final result, the following steps, data synthesis, intention to treat analysis, binary data, continuous data and test of heterogeneity are also included in the study. The study leads the reviewers to the final result, that is, “no randomised results controlled trials that fulfilled the inclusion criteria.

Read More