Critical Appraisal Of Qualitative and Quantitative Research Articles - Essay Example

Critical Appraisal Of Research Articles Abstract This Paper provides a critical appraisal of two scholarly articles that have been presented on research methods. The aspects governing the qualitative research method have been dealt with in an article analyzing qualitative research by J.V. Appleton, while the quantitative research method was applied in a study conducted by Shuster and Shannon. Both these articles identify the potential limitations that may be inherent in such research methods and present ways and means to enhance the reliability and applicability of data gathered from a limited sample to a wider population. Critical appraisal of two research articles The article titled “Analyzing Qualitative interview data. Addressing issues of reliability and validity” presented by J.V. Appleton deals with the aspects that must be taken into consideration while conducting a qualitative research study to ensure the reliability and validity of the data (Appleton, 1995). In contrast, Shuster and Shannon (1994) in their article titled: “Differential Prehospital Benefit from Paramedic Care” present the aspects affecting the efficacy of quantitative research methods. Both article authors have based their findings upon the results of actual studies they have personally carried out in order to assess the relative efficacy of either method. The two articles present valuable insight into the process of conducting research and the means that may be employed to ensure that data obtained are as valid and reliable as possible. Both these studies were conducted using health professionals as the source of the data. Summary of Appleton’s article: J.V. Appleton (1995) is a heath care visitor/worker and the aim and objective of her research effort was to identify a set of criteria to identify and determine which families could be considered “vulnerable” from the point of view of child protection. The method chosen for her research study was a qualitative approach based upon a two fold process – the administration of questionnaires as well as follow up interviews. The reason for the choice of a qualitative approach lay in the fact that this was an exploratory study designed to elicit the perspective of health care workers. Therefore the quality of the data was dependent upon the richness and quality of the responses that needed to be obtained rather than upon statistical numbers and figures that could be used as the basis for the study. Qualitative research has been found to be useful in the study of exploratory phenomena (Morse, 1991). Appleton’s study was based upon questionnaires administered to 12 health care workers followed up by face to face interviews, with a tape recorder being used in all cases. The questionnaires functioned as a back up source of data which was used as a means of verification with the results obtained from the interviews, which were designed to “elicit detail” as specified by Lofland (1971). The quality of data that is collected during an interview depends to a large extent upon the skills and expertise of the interviewer, who functions as the control factor that determines how much at ease the respondents are and how valid is the quality of their response. In order to reliably analyze the data, Appleton describes how she sifted through the data and abstracted it in order to draw out the salient points that had emerged from the vast amounts of data that were collected, using a three stage data analysis method identified by Miles and Huberman (1984) – data reduction, data display and data drawing. She categorized the interview responses into sub headings, which were then collated with the responses slotted into the appropriate categories. This enabled later pooling of data followed by comparison with the data collected from the questionnaires in order to arrive at the results that she obtained. Through this process of analysis, Appleton was able to distill six basic themes prevailing in all the data: (a) Vulnerability – the ambiguous term, (b) vulnerability – a complex mix of factors (c) The continuum of vulnerability (d) professional judgments vs official guidelines (e) the health visitor-Information coordinator and (f) Role diversity and conflict. Appleton identifies four critical parameters that determine the validity of the results. (a) truth value (b) applicability (c) consistency and (d) Neutrality The truth value of a research study pertains to how closely it mirrors actual fact, and Sandelowski (1986) states that a research instrument may be said to be valid when there is a level of confidence imbued in the study that it does in fact measure what it sets out to measure. The credibility of a research study will therefore be determined by how accurately it describes peoples’ experiences and how closely people who have indeed been through that experience would identify with the results. Applicability of research results on the other hand, refers to how representative the results are and how broadly it can be generalized to fit across a larger population sample. Miles and Huberman (1984) have identified the threat of “holistic fallacy” that can result from improper weightage of data and by too much reliance upon responses obtained from well informed, articulate respondents. In order to avoid the kind of overconfidence and bias that results as the researcher become more sure of results, Appleton employed the technique of repeatedly checking back with data from the questionnaires to ensure applicability and validity of the data in question. The consistency of research results is a measure of how often the measures can be repeated to obtain consistent results. Since qualitative data in itself measures the uniqueness of human situations, it is important that they must be consistent in order to ensure validity. In order to be consistent, a study needs to have a “decision trail” and an audit trail, which emphasizes how effectively, the researcher’s methods can be evaluated in order to discover the pattern he or she has used (Sandelowski, 1986). Appleton was able to adhere to this requirement through the provision of adequate quantities of information about the data collection and collation process, thereby ensuring consistency. Reliability of data collected was also ensured through the use of a tape recorder for all respondents, use of the same questions in the same context and by attempting to duplicate conditions of interview wherever possible. Neutrality of a research study is determined by the lack of bias on the part of the researcher and Appleton attempted to address this requirement by maintaining a neutral stance throughout the process, in spite of being a health worker herself. Summary of article by Shuster and Shannon: This article addresses the quantitative method of research, as opposed to the qualitative method detailed by Appleton. Quantitative research is based upon the collection of statistical data that already exists rather than an exploratory view of data collection. The aim and objective of this research study was to determine whether there was any difference in the outcome of hospital emergency patients depending upon whether they received treatment from MET-P specialists trained to react to all kinds of emergencies or MET-D specialists trained primarily to administer defibrillation procedures in cardiac cases. It is difficult to effectively and reliably measure this difference owing to the fact that possibility of other long term complications exists in the patients and conditions during an emergency are not really conducive to the process of data collection and experimentation in a scientific setting. Therefore, the measure that was used in this study was based upon the input of the health professionals rather than the patients. The data source in this study was the 7-point Patient Severity Rating Scale that the Paramedics filled out at initial contact and before hospital technicians take over. This scale rated the patients’ vital signs and assigned a “problem classification” designating the kind of trauma or crisis that the patient was going through. The control that was used in this instance was that of the second opinion received from a medical practitioner at the hospital by way of independent assessment of the patient’s condition upon arrival and completion of the 7 point severity scale. In qualitative studies, the volume of data collected must be adequate in order to provide statistically reliable results. In this case study, the number of patients assessed was 10,291 and the paramedics were encouraged to complete the scales, with missing data being disqualified from inclusion in the study results. In this way, this study was able to provide a control factor as well as adequate amounts of reliable data on the basis of which to arrive at conclusions. Therefore, this study was able to address to a large extent, any bias of the medical assessment provided by paramedics as well as the elimination of irrelevant or missing data that would have interfered with the validity of the results obtained. However, there were several limitations to this study, not least of which was the fact that patient outcome is difficult to measure in an emergency setting, especially from lack of knowledge of other long term factors that may affect the patient’s condition. When there are such a multiplicity of factors that may affect a patient’s outcome, it is difficult to test exactly to what extent status change in the patient’s condition can be measured using the assessment of the paramedics. However, using the parameter of status change, three categories were distinguished: improved, unchanged and worsened. The separate “problem classification” category that was assigned helped to determine the nature of the condition and whether it was critical or moderate. The results that were obtained from this study showed that there was no significant change in status that could be attributed to the paramedic care when the patient’s condition was not so severe. However when the patient’s condition was critical, the MET-P specialists were found to elicit better recovery rates and outcomes as compared to MET-D’s who were better equipped to handle only certain categories of emergencies that were related to critical cardiac conditions. The results that were obtained were correlated with multiple tests upon the same data and it was found that significant results of correlation existed between the various test results, thereby providing the assumption that the study results that were obtained were fairly reliable and valid. Even with the use of a Bonferroni correction, significance level remained at 5%. The biggest limitation identified in this study was that of rater bias. The assessment of improvement in condition and/or no change/deterioration was left to the discretion of the assessor and results were rarely cross checked unless there was an obvious and significant level of error in the assessment that became evident during hospital care. The efficacy of this pre hospital care could not also function as a reliable indicator of whether or not the patient’s stay in the hospital would have a favorable outcome. Analysis of validity and reliability of the two studies: In the above analysis, the measures that were taken by the researchers to improve levels of reliability and validity of their studies was outlined. However, the effective assessment of the efficacy of their respective studies may be made through the input of Greenhalgh (1997), who identifies the following questions to be posed while reviewing a research study in order to determine its reliability and validity. (a) Was the study original – does it add to the already existing literature? Both the research studies identified above are original in that they contribute a fresh perspective to already existing literature on the subject of identification of vulnerable families and assessing the differential in patient outcomes from pre hospital care. Both these studies are more rigorous than others in that efforts have been made to address criticisms of methodology that have arisen in other studies. (b) Whom is the study about? In the first case, the study used health care workers to identify vulnerable families and in the second case, the study used paramedics to assess the condition of patients. Shuster and Shannaon have taken pains to exclude all data sources that were missing or incomplete, while Appleton has endeavored to collect comprehensive information by using two research methods – questionnaire and interview. However, both these studies show an important limitation in this area, in that assessment of “vulnerability” and “improvement” have been made by the health workers themselves and this introduces the element of bias. For example, Appleton’s bias would have invariably filtered down into the results just as the bias of the paramedics would have filtered down into assessment of improvements in the Shannon and Shuster study. Therefore, this would introduce the element of doubt into the results of both the studies. (c) Was the design of the study sensible? Appleton’s study has been rigorously designed to evaluate qualitative with as great a degree of reliability as possible. Similarly, Shuster and Shannon’s study also makes use of scales that have been pre-tested in other studies and found to be a reliable measure of a patient’s condition. Therefore, all effort has been made to undertake sensible studies. However, when Greenlagh‘s(1997) questions – what was the outcome of the study? Is applied to Shuster and Shannon’s study, it will be found that the outcome was indeterminate or undefined. In order for the efficacy of pre hospital treatment to be reliably measured, it is also essential to introduce some parameters to measure the final outcome of the patients after their hospital stay. It is only in conjunction with this information that the relative efficacy of pre hospital treatment can be effectively measured so that its results can be uniformly applied across the spectrum. (d) Was systematic bias avoided or minimized? In this aspect, as identified above there was every effort to minimize bias in Appleton’s study, however the only measure of this degree of impartiality lies in Appleton’s statements, which may not actually be the case in practice. In Shuster and Shannon’s study, bias is one of the most important limitations. However, in terms of randomized, controlled trials, it may be noted that Appleton’s study was limited in that there were only 12 respondents since the interview method involved considerable time, but results obtained could have been more reliable and applicable across a wider spectrum if a wider respondent base had been identified. Alternatively a control group could have been selected to provide a basis for comparison rather than using the results from questionnaires obtained from the same category of respondents. In the Shannon study, systematic bias was tackled by eliminating missing or incomplete data. Conclusion: In conclusion, it may be stated that on an overall basis, both the research studies have been undertaken with every effort made to ensure reliability and accuracy in the results. However, the applicability of Appleton’s study over a wider framework remains to be further tested before conclusions can be drawn, although results were tested across some population samples. Her study is hindered by the small size of respondents. In Shannon and Shuster’s study, the primary limitation that may be identified lies in the bias of the paramedics and the fact hat no effort was made to cross check the validity of the paramedic assessments by providing control medical personnel in attendance at the emergency site to evaluate the same symptoms independently. Therefore different results could be obtained in other population samples. Moreover, this study is also limited by the fact that it is lacking in an important factor – the assessment of final outcomes at the hospital, because the purpose of such a prehospital assessment study would be undertaken only to evaluate contribution to final outcomes. References cited: * Appleton, Jane V. (1995). “Analyzing Qualitative interview data: addressing issues of validity and reliability”. Journal of Advanced Nursing, 22, pp 993-997. * Lofland, j. (1971). “Analyzing social settings: A Guide to Qualitative observation and Analysis”. Belmont, California: Wadsworth. * Miles, MB and Huberman A. (1984) “Qualitative Data Analysis: A sourcebook of new methods.” London: Sage. * Morse, J.M. (ed) (1991). “Qualitative nursing research. A contemporary dialogue” London: Sage. * Sandelowski, M. (1986). “The problems of rigor in qualitative research”. Advances in Nursing Science, 8 (3), pp 27-37 * Shuster, Michael and Shannon, Harry S. (1994). “Annals of Emergency medicine”, 23(5) Read More