Crime Prevention - Book Report/Review Example

CRITICAL EVALUATION OF RESEARCH STUDY 2007 CRITICAL ANALYSIS OF RESEARCH STUDY Gill, M and Spriggs, A (2005). Assessing the impact of CCTV. Home Office Research Study 292, London, Home Office The study examines in detail the issues associated with effective operation of Closed Circuit Television Camera (CCTV) systems. Gill and colleagues clearly state the major purpose of their study, namely: "to measure the impact of the CCTV project on crime and fear of crime" (p. vi). However, analysis of the study aims, findings, and the methodologies and research techniques used by the authors suggests that the basic objectives may also include the following: to provide highly detailed description of the impact of CCTV on behaviour of individuals and fear of crime; to reveal the characteristics of CCTV systems' operation that affect effectiveness/ineffectiveness of CCTV projects; to perform economic evaluation of CCTV projects. The purposes help determine the relevance and importance of the authors' effort. Firstly, the report represents a major advance in our knowledge about CCTV, especially if consider absence of previous research exploring the issue so comprehensively. Secondly, the study offers a number of valuable implications for CCTV projects: it helps understand the different aims of CCTV, and the mechanisms by which those aims can be realised; it helps formulate a theory of how CCTV should operate to ensure the highest effectiveness, and a hypothesis for the evaluation of CCTV effectiveness. The authors do not postulate a hypothesis to be tested, but it can hardly be considered a drawback. The report evaluates 14 projects implemented in different urban contexts, and it is difficult, if possible, to formulate a common hypothesis to fit all the cases. However, the implicit hypothesis underlying the study may be the assumption that CCTV could be effective in tackling crime and fear of crime under certain conditions. The measurements and evaluation performed using different tools help the authors test this assumption. The introductory section of the study includes a solid review of relevant literature. The authors list the most credible research in the field and identify gaps in the existing body of knowledge in order to justify the need for their own efforts. The review is divided into several sections and effectively provides the background for the study with reference to the formulated objectives. Smart synthesis and analysis of the past research findings coupled with the authors' ability to highlight the most interesting of them helps the authors properly link findings of their study to a wider framework of theoretical knowledge. METHODOLOGIES Methodological framework employed by Gill and colleagues for their study deserves particular attention. The authors used two approaches for the purposes of their study: quasi-experimental research techniques and realistic evaluation. Evidently, such combination of research methodologies is employed to reinforce validity and reliability of the findings: the authors attempt to take advantage of the strengths of each method using them in such a way to address the potential threats in the most effective way. Quasi-experiment is a popular method of quantitative research. Quasi-experimental studies bear much resemblance to the true experimental design with only one difference: in a quasi-experiment the researcher assign participants to different groups non-randomly. This research method has been developed as an alternative tool for examining causal relationship in situations that, for some reason, are poorly conducive to experimental control. This is exactly the case in studying the change in crime and fear of crime following the installation of CCTV. The most common characteristics of quasi-experiments are the following: Use of matching instead of randomisation; Use of interrupted or uninterrupted time series analysis (a type of longitudinal research); Quasi-experiments are more effective than the true experiment in analysing contextual concepts (e.g. quality of life, morale, etc) A subject variable as an independent variable: the researcher has to select rather than vary the independent variables (Cook and Campbell, 1979). The quasi-experimental study design also known as 'before-after intervention' study design, is often used to evaluate the effectiveness of specific interventions such as, for example, CCTV project. Realistic evaluation represents a different research paradigm. The major assumption underlying the orthodox experimental design is that similar combination of causes will always produce similar results. Proponents of the realistic evaluation approach question this 'constant conjunction' account of causality and suggest a different view. From this perspective causality is explained in terms of underlying - and often hidden - causal mechanisms generating regularities (Pawson and Tilley, 1997). Although the term 'realistic evaluation' suggests that this approach relies upon the philosophy of realism and, therefore, belongs to the quantitative methodologies, it is qualitative in nature. Realistic evaluation approach seeks to understand and explain phenomena in a specific context, namely: "explain how the particular initiative [CCTV] had worked, paying particular attention to the circumstances in which it was installed" (p. 123). Similarly, qualitative methods explore phenomena in "real world setting [where] the researcher does not attempt to manipulate the phenomenon of interest" (Patton, 2002: 39). While experimentation employs a range of statistical instruments and other tools to obtain quantifiable data, realistic evaluation is not concerned with quantification in the first turn. Instead, it allows the research to understand the underlying hidden factors that determine success/failure of an intervention under specific set of circumstances. APPROPRIATNESS OF STUDY DESIGN The choice of combined methodological frameworks seems reasonable and appropriate due to several considerations. Firstly, broad research objectives pursued by the authors cannot be achieved unless several different techniques are used. Thus, some of these objectives (e.g. to measure the impact, to provide economic evaluation) are possible to achieve only with the help of quantitative tools; others (e.g. to reveal the characteristics/mechanisms, to provide a detailed description) require use of qualitative approaches. Quantitative tools have dominated the area of professional research in many fields of knowledge including criminology, and even the recent surge of attention toward the qualitative perspective hardly threatens their dominant position in this field. With the help of these tools the researchers are able to obtain objective quantifiable data, establish causality, evaluate, etc. Quantitative approach, which implies use of standardized questionnaires and methods of data analysis, specific position of the researcher (considered external to the research), and replicability of the results regardless of the context has been traditionally considered to be more objective than its qualitative counterpart (Wainer and Braun, 1998). Secondly, specific nature of the formulated research purpose raises a set of strict requirements to be addressed: identification and evaluation of factors, which play certain role in the prevention of crime, call for applying rigorous instruments. Although quasi-experiment does not imply as much rigour as, for example, Randomised Control Trial (RCT) (Kaplan, 2004), the level of rigour and reliability secured by this technique is sufficient. At the same time, RCT involves random distribution of individuals among a control group and intervention group. This is more expensive and time-consuming than quasi-experimental design, which also involves comparison between a control and intervention groups but without random selection. In the combination of methods used by the authors the primary role perhaps belongs to the quantitative techniques, while the qualitative assessment plays supplementary, though also important role. Such lack of balance can be justified by the existing body of knowledge about CCTV and its role in crime prevention. If the knowledge base were relatively little than it would probably be prudent to employ qualitative tools as the primary source of information: qualitative methodology is exploratory in its nature (Creswell, 1997: Patton, 2002). As our level of knowledge increases, rigour, validity, availability of control and other characteristics of quantitative methodologies become a more appropriate alternative: qualitative methodologies are characterised by "lack the rigor and objectivity which characterise the quantitative approach" (Patton and Appelbaum, 2003: 60). Therefore, quasi-experimental (before-after) design is better suited to evaluating the effectiveness of CCTV intervention. On the other hand, qualitative evaluation techniques, which have "had a rough time gaining acceptance in the mainstream social and behavioural science research" (Boyatzis, 1998: vi), help 'contextualise' the quantitative data and explain the differences observed in effectiveness of similar CCTV projects implemented in various environments. Due to variations of the contexts and hidden mechanism CCTV will unlikely produce the same effect on crime rates in all circumstances. Besides, the nature of some findings listed by Gill and colleagues demonstrate that the qualitative data played the key role in revealing them. Thus, the list of factors that influence CCTV's operation in preventing crime shows that: The researchers express an interest in understanding the experiences perceived by the participants of their study and strongly emphasize the context in which the perceptions take place. The study involves in-depth exploration of all aspects related to CCTV's operation with the authors requiring from the participants as comprehensive information as possible. The type of reasoning utilised by Gill and colleagues is predominantly inductive. As it has been mentioned, the authors fail to formulate a hypothesis or suggest a theory to be tested/verified: it is evident that the list of finding has emerged from the data gathered during the evaluation. And finally, discussing effectiveness of CCTV the researchers admit that "research on mechanisms has adopted a scientific realism approach, rather than a quasi-experimental one [and] Knowing how CCTV works is vital for developing 'transferable lessons' that enable good practice in one area to be repeated in another" (p.6). Transferability is the criteria that is largely ignored by quantitative methodologies, and favoured within qualitative research (Winter, 2000; Hoepfl, 1997). DATA COLLECTION AND ANALYSIS Data gathering techniques preferred by the authors within both methodologies are congruent with the purpose of their study. Quasi-experimental (before-after) design implied collecting information about the initial status of a group/area in terms of the outcomes of interest, and then collecting information about the outcomes after intervention (CCTV system installation) was made. Data recorded for the 'control' group helped the authors judge whether the intervention alone was responsible for the changes, if any, in the outcomes. Police-recorded crime statistics is the principal source of data, though census data (socio-demographic) is also used together with public attitude surveying before and after intervention. Data collected during the surveys are especially important in measuring changes in levels of fear of crime. The authors also employed a wide range of qualitative methods such as interviewing (unstructured and semi-structured), participant observation, study of documentation and focus groups, though interview was the primary tool. Interviewing has certain advantages and disadvantages as compared with other qualitative methods such as participant observation or focus groups (Creswell, 1997). The advantages of interviews include ability of the researcher to involve participants whose viewpoints, status, experiences and/or knowledge of the issue being studies match the purposes of research best. The nature of interviews allows the researcher to retrieve as much information on the issue being studied as possible (Glesne and Peshkin, 1992). Therefore, the interview data add greatly to the understanding of the issues identified by a survey or experiment. While experimental approach was instrumental in identifying certain variables and/or relates them to each other, the interview helped Gill and colleagues understand the meaning of these variables within a context-specific setting. Other qualitative techniques performed the same function. For example, focus groups were carried out in order to "explore how the presence of the CCTV camera(s) actually affected the people living in these areas" (p.133). Although the authors do not provide detailed accounts of strength and limitations of each technique used, the do, however, present an explanation of how the quantitative data obtained is treated and interpreted. Numerous tables provided throughout the paper make the analysis more structured and facilitate comparisons. The authors also cite a number of highly relevant case studies to illustrate the most important findings. While analysis of quantitative data always follows standard procedures, analysing qualitative data is a more difficult task. Analysis of qualitative data requires the ability to manoeuvre across the wide range of options in order to as accurate analysis as possible in each particular case. Gill and colleague do not provide any information about the specifics of qualitative data analysis. Thus, it is not clear whether each interview is tapped, transcribed and read several times; the authors do not describe any well-established procedures for the rigorous analysis of data; no coding scheme designed with reference to the study's objectives are mentioned. The study does not include any information concerning whether the authors employed any specific software such as SPSS (statistical measures) or NVivo (qualitative data) to analyse, correlate and interpret data. The huge volume of various data processed by Gill and colleagues suggests that they did take advantage of specific computer programs: acknowledging this fact by naming the software might be useful as well. BIASES AND LIMITATIONS The authors seem to be fully aware of the major biases and limitations of their study and acknowledge this fact by listing the most important of them. Evidently, the most important concern for Gill and colleagues is securing internal validity of the study. The overall purpose of the authors in this regard is "achieving where possible Level 3 of the Maryland Scientific Methods Scale" (p.20). The Maryland Scale of Scientific Methods ranks a study on the five-point scale with the range from 1 (weakest) to 5 (strongest) depending upon the overall internal validity of the study (Sherman et al, 1998). Different settings and contexts require different criteria and methods, but regardless of the setting, the Maryland Scale implements some common core criteria to distinguish between the levels: Level 1. Correlation between a crime reduction/prevention programme and a measure of crime or crime risk factors at a single point in time; Level 2. Temporal sequence between the project and the crime or clearly observed risk outcome, or the presence of a comparison group without demonstrated comparability to the intervention group; Level 3. A comparison between two or more comparable units of analysis, one with and one without the intervention; Level 4. Comparison between multiple units with and without the program, controlling for other factors, or using comparison units that evidence only minor differences; Level 5. Random assignment and analysis of comparable units to program and comparison groups (Sherman et al, 1998). One might think that the basic characteristics of quasi-experiment listed in the previous section allow the authors not only achieve Level 3, but also exceed it. However, Gill and colleagues do not claim any higher level, which is the best evidence they are perfectly aware about the internal validity threats impossible to overcome in their study. Firstly, although the method of quasi-experiment has been traditionally viewed as superior to non-experimental approaches it has some important limitations. Some of them are common to other experimental techniques (e.g. true experiments), but others characterise only quasi-experiments: Limited potential of the experimental data in answering some questions which may be of certain interest to the researcher. Thus, no parameters that depend on the joint distribution of outcomes in the control and experiment groups can be estimated: "Only if the evaluation problem is defined exclusively in terms of means can it be said that experiments provide a precise answer" (Heckman and Smith, 1995: 22). Experimental studies only help evaluate and assess, but provide no explanation why some or other results have been revealed (Kaplan, 2004). Quasi-experiment lacks the control features of true experiment. This results in absence of control over external variables and lower internal validity: the researcher does not have the same confidence in revealed causal relationships as in true experiment (Cook and Campbell, 1979). Evidently, Gill and colleagues are specifically concerned about the lack of control features. This concern is expressed in the beginning of the study in the discussion of CCTV systems impact on crime: "only two [CCTV systems] showed a statistically significant reduction relative to the control, and in one of these cases the change could be explained by the presence of confounding variables (p.6). Evidently, confounding influence of such variables is impossible to eliminate within the framework of quasi-experiment. This is perhaps one of the major reasons for Gill and colleagues claiming to achieve the third level of the Maryland Scale, not the forth or fifth. Secondly, the factor known as 'statistical power' also affects the study's effectiveness in discovering the impact of intervention. This factor comprises several components: base rate of crime, sample size, etc. Only when all these components are properly addressed the researcher may be sure that the study reveals a true before-after intervention difference (Sherman et al, 1998). Gill and colleagues openly admit that some of these components have not been addressed. Thus, in the section dealing with measurement of CCTV systems' impact on crime levels they note that "small sample sizes render [the following] findings merely suggestive" (p.58). The authors' professional qualification coupled with the fact that they recognise importance of the statistical power suggests that there were practical considerations (e.g. lack of time, funding constraints) that should be held responsible for lack of statistical power. Thirdly, any study involves certain external (not related to the method employed) factors that may undermine internal validity of any research. The sum of such factors (e.g. low response rates, numerous drop-outs, etc) causes the so-called 'measurement error' (Sherman et al, 1998). Measurement error seems to be another important factor that prevented Gill and colleagues from reaching a higher level on the Maryland Scale. The authors admit that three major factors, namely the difficulty of identifying a suitable control for each target area, unavailability or poor quality of police crime data, and CCTV system implementation failures in some areas: "the work was impeded by implementation delays and failures, which in some cases led to schemes being dropped from the evaluation, and lack of access to data, which in some cases rendered the analyses less robust than had been hoped" (p.9). The combination of these factors negatively affects quality of data, and some projects under evaluation have no comparable control: crime rates in such cases are compared with those in the division, though Gill and colleague claim "the results in this study give little indication that a control area provides a better comparison than the division" (p.20). The Maryland Scale lists the following internal validity threats which are progressively eliminated up the scale: Causal direction: the question of whether the crime caused the programme to be present or the program caused the observed level of crime; History: long period of time or other variables external to the programme that may have caused a change in crime rather than the prevention program itself; Chance factors: either within the programme group (for example, several potential offenders are put to prison) or external events (for example, unusual weather conditions in the area). These unexpected variable may be the true cause of any measured change in crime; Selection bias: the opposite of 'randomisation bias' - a set of variables, which characterise the group receiving a programme, that independently affects the observed level of crime (Sherman et al, 1998). The essence of the Maryland Scale is that the only way for a practitioner to demonstrate the causal relationship existing between the programme impact and crime level/fear of crime is to eliminate as many of these threats as possible: ideally, all of them (Cook and Campbell, 1979). Thus, a study ranked at Level 5 must effectively control all these threats, and a study ranked at Level 1 does not control any of the four. Claiming that their study achieves Level 3 Gill and colleagues assume that they properly control at least half of the above list of threats. Chance factors seem to be one of the threats the authors do not control. Thus, they are not able to determine whether the difference between pre- and post-implementation crime rates is due to positive impact of CCTV systems or chance factors: "Even where changes have been noted very few are larger than could be due to chance alone and all could in fact represent either chance variation or confounding factors" (p.43). Similarly, the history threat is not properly addressed: the authors admit that quality of some evaluations suffered seriously due to implementation delays. Selection bias does not undermine internal validity of the study. Gill and colleagues claim that they were not involved in selecting the CCTV projects, but the process followed three different criteria to obtain a representative mix characterised by "a range of different settings" (p.13). Causal direction has been addressed by the quasi-experimental approach. And finally, qualitative methods (interviewing, focus groups, participant observation) also have potential constraints and limitations. Furthermore, they are even more difficult to address and control than the threats inherent in quantitative research. Thus, data obtained during interviewing is believed to be subjective and descriptive (Murphy et al, 1998). Probably the main drawback of this method is related to the language-imposed constraints: the researcher's questions asked verbally during the interview are believed to frame responses of the participant being interviewed (Hancock, 1998). MEASURES TO REDUCE BIASES AND LIMITATIONS Being perfectly aware about the limitation and biases of the study, the authors take some measures to reinforce credibility and trustworthiness. For example, Gill and colleagues smartly address the problem of one potentially confounding variable (street lighting) by treating it as part of the scheme "rather than as confounding factor" (p.2). The popular strategy of 'triangulation' is also found in the study. Triangulation is widely employed to enhance trustworthiness of research by eliminating the biases (Massey, 1999): Methods triangulation - several different strategies of data collection are employed (e.g. public attitude survey and interviewing); Sources triangulation - the thematic findings are triangulated with the literature review. However, the authors do not pay much attention to triangulation. Perhaps in doing so they are guided by practical considerations as well: given absence of a hypothesis to be tested objectives of the study are different, and reached with the help of different methods. On the other hand, researcher triangulation might be useful even in such situation. This type of triangulation implies that the research involves several researchers to analyse the data, develop and check the coding scheme (Massey, 1999). For example, researcher triangulation could improve credibility of qualitative data obtained via interviewing. CONCLUSION The study under review represents a solid piece of work written by the top-level professionals in the field of crime prevention. In terms of the research methodology, the authors provide a good example of how quantitative and qualitative approaches may be effectively used for the purposes of one study. Thus, Gill and colleagues illustrate the opinion that the philosophical distinctions between quantitative and qualitative research epistemologies are not as stiff as many believe, though some researchers still question the possibility of accommodating them within one study (Guba and Lincoln, 1994). The authors provide a strong synthesis of the research data and major findings. The key finding of the study is absolutely congruent with the objective stated in the very beginning: only two systems showed a statistically significant reduction in recorded crime. However, the list of secondary findings is impressive including: impact of CCTV on different types of crime, change in public attitudes to CCTV system, factors that influence CCTV operation, and cost-benefit analysis. The authors are fully aware of the biases and limitations that are impossible to overcome within the chosen methodologies, and they explicitly acknowledge this by listing them. The choice of methods and data collecting techniques perfectly fits the objectives the researchers want to achieve. A combination of quantitative and qualitative methods provides the authors with an opportunity to broaden the scope of their evaluation, and represents a nice example of how the different paradigms relying upon different epistemological foundations can add to each other. REFERENCES Boyatzis, R. E. (1998). Transforming qualitative information. London: Sage Cook, T. D, and D. Campbell (1979). Quasi-experimentation: design and analysis issues for field settings. Chicago: Rand McNally Publishing Creswell, J. W. (1997). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage Publications Gill, M and Spriggs, A (2005). Assessing the impact of CCTV. Home Office Research Study 292, London, Home Office Glesne, C., and Peshkin, P. (1992). Becoming qualitative researches: An introduction. New York, NY: Longman Guba, E.G. and Y. S. Lincoln (1994). Competing paradigms in qualitative research. In: N.K. Denzin and Y.S. Lincoln (Eds.). Handbook of Qualitative Research, Thousand Oaks, CA: Sage, 105-117 Hancock, B. (2002). An Introduction to Qualitative Research. Trent Focus Group Heckman, J.J and J.A. Smith (1995) Assessing the case for social experiments, Journal of Economic Perspectives, Vol. 9(2), 85-110. Hoepfl, M. C. (1997). Choosing qualitative research: A primer for technology education researchers", Journal of Technology Education, 9(1): pp.47-63 Kaplan, D. (2004). The SAGE Handbook of Quantitative Methodology for the Social Sciences, London, Sage Massey, A (1999). Methodological Triangulation, Or How To Get Lost Without Being Found Out. In: Massey, A. and Walford, G. (Eds.) Explorations in methodology, Studies in Educational Ethnography, Vol.2, Stamford, JAI Press, 183-197 Murphy, E., R. Dingwall, D. Greatbatch, S. Parker, and P. Watson (1998). "Qualitative research methods in health technology assessment: a review of the literature", Health Technology Assessment, Vol. 2: No. 16: pp.141-165 Patton, M. Q. (2002). Qualitative evaluation and research methods (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc Patton, E. and Appelbaum (2003). The case for case studies in management research, Management Research News, 26, 60-62 Pawson, R. and Tilley, N. (1997). Realistic Evaluation, London: Sage Sherman, L.W., Gottfredson, D.C., MacKenzie, D.L., Eck, J., Reuter, P., and Bushway, S.D. (1998). Preventing crime: What works, what doesn't, what's promising. National Institute of Justice: Research in Brief. Washington, DC: US Department of Justice, National Institute of Justice Wainer, H., and Braun, H. I. (1998). Test validity. Hilldale, NJ: Lawrence Earlbaum Associates Winter, G. (2000). "A comparative discussion of the notion of validity in qualitative and quantitative research", The Qualitative Report, 4(3 and 4) [available online at http://www.nova.edu/ssss/QR/QR4-3/winter.html] Read More