Impact Evaluation Issues - Essay Example

Introduction Impact evaluation is defined by Baxter (2004, p.33) as the assessment of the changes which result from a specific intervention. The social change which is attributed to a program, project or policy can be evaluated to determine its success. In social change, impact evaluation assesses outcomes such as positive changes in a social group which promotes its well being. This is done by evaluating the changes that have occurred against what the situation would have been if the intervention was not taken. Therefore the impact evaluation is not a mere monitoring of outcomes but deep analysis of the results of an intervention so that the effects are accurately measured. The changes which an impact evaluation seeks to assess must be directly related to the project or program which resulted to the changes. Impact evaluation is therefore important in making various decisions and policies. Impact evaluation has been used widely in determining the most appropriate and effective approaches for promoting the well being of the society. Moreover, programs in the social sector are usually assessed through the impact evaluation methodology. This paper is a case study that gives an analytical explanation of what comprises of good impact evaluation in the context of intervention for social change in relation to the impact evaluation of Puntos de Encuentro’s Communication Strategy in Nicaragua on Catalyzing Personal and Social Change around Gender, Sexuality and HIV. In social change, impact implies a long term effect on the society. Solorzano and Bank (2008, p.12), explain that the effect may be unintended, intended, negative or positive. A good impact evaluation must focus on the relevance of the intervention. To determine how relevant a program or project is, its objectives are assessed to determine if they are right or not. The SDSI strategy of Puntos de Encuentro’s main objective was to prevent more infection of HIV in Nicaragua. The intervention that the strategy employed in the case study is the use of the mass media and its communication programs to reach the community. This strategy is relevant to Puntos de Encuentro’s objective because it is through communication to the affected members of the society that that health education and general awareness to the society would be made possible. Deacon (2007) illustrates that in health programs, objectives such as the empowerment and promotion of societal rights on issues such as violence, reproduction, drug abuse and HIV are used to do an accurate impact evaluation of the effectiveness of such programs or projects. In addition, Bradshaw , Solorzano and Bank (2006) emphasize that it is through the objectives of a project that its relevance is determined and therefore a good impact evaluations must be able to assess whether a program for social change is relevant to the intended goal or not. According to Baxter (2004, p.39), a good impact evaluation should be able to determine the efficiency and effectiveness of a program or project to enable decision makers to come up with proper policies which would promote the intended social change within a specific society. The effectiveness of the Puntos de Encuentro program is illustrated by the tools used for the achievement of the objectives. The use of edutainment programs which involves educating the society through entertainment demonstrates the efficiency of passing the messages to the community as compared to formal education programs. The use of coordination and links within the community was aimed at making the program as effective and efficient as possible. Moreover, the program formed alliances within the community which facilitated the education of the community of matters of gender, sexuality and HIV/AIDS. Solorzano and Bank (2008, p.35), illustrate that interventions such as networking, communication of the leaders and mobilizations are effective in the realization of social change. The impact of a given change is important in determining whether the program is successful or not. In social change, a positive impact of an intervening project is usually expected. Good impact evaluations thus should be able to tell whether the resulting effect of the program yields positive or negative changes in relation to the expected outcomes. Moreover, the impact of a social change should be sustainable. As a result good impact evaluations must be able to assess the sustainability of the change. The Puntos de Encuentro program aimed at causing a positive social impact on matters of sexuality, gender and HIV. The impact felt was the awareness of the society on these issues and the reduction of the spread of the HIV infection within the Nicaragua region. The impact of the program should also be assessed to find out if it will be sustainable in causing the intended social change. Deacon (2007) explains that in impact evaluation, a project is assessed in terms of its benefits and whether they are sustainable. Impact evaluation uses various methodologies to assess social interventions to find out if they are appropriate to particular desired change or not. Asselin and Parkins (2009, p. 483) point out that good impact evaluation must be able to measure the design used by the project in selecting samples and collecting data to find out if they are appropriate or not. The Puntos de Encuentro program used a project design that employed a communication model in programs within the mass media. The evaluation of the SDSI used qualitative and also quantitative instruments which included surveys of households in the community and interviews. According to Solorzano and Bank (2008, p. 14), qualitative and quantitative methods or both are used during impact evaluation to assess projects and programs intended for social change. In social programs, qualitative techniques are commonly used because they are suitable in exploring and providing a wide range of answers to mistakes in the designs of the programs. Asselin and Parkins (2009, p. 487) asserts that indicators such as benefits of a project must be considered in performing an impact evaluation to determine if the benefits of the project outweigh the costs incurred. The cost-impact ratio shows that the SDSI intervention led to a greater contribution to the prevention of HIV in the Nicaragua region as compared to the incurred cost of the program. Weston (1998, p. 781), emphasizes that good impact evaluation must be able to determine if an intervention for social change is worth the amount of resources put into it and if it is feasible. According to Asselin and Parkins (2009, p. 492), the verification of the changes which result from a specific program or project is part of an impact evaluation process. Solorzano and Bank (2008, p.21), explain that a good impact evaluation should verify the changes by ensuring that the evaluation process is implemented. The implementation of the impact evaluation must be preceded by the creation of a proper implementation plan. A plan ensures that the changes of a specific social program are verified. Deacon (2007) illustrates that an impact evaluation plan includes an outline of the data to be gathered in addition to the place and people involved in the data collection. A good impact evaluation plan thus must include the preparation for data gathering such as the tools to be used. Additionally, a good impact evaluation should have a plan for recording and analysis of data so that the changes of a project are determined or verified by the evaluation process. According to Weston (1998, p. 781), the outcomes of interventions for social change comprise of the ultimate impacts of programs or projects on the society and therefore good impact evaluation must assess these outcomes. The outcomes of a project are necessary in finding out whether the project was worthwhile and if it led to the intended change. For example outcomes in social health programs such as promotion of the quality of life are important measurements of an impact evaluation process. Such outcomes are the determinants of a program and as a result good impact evaluation must assess them to illustrate the success of the program. The Puntos de Encuentro program resulted to the desired results of significantly preventing the HIV infection within the community. Thus in the evaluation of this program, the outcomes are used to provide a report which would guide decision makers in reaching decisions and policies which determine the process of further social change. Baxter (2004, p. 44) points out that good impact evaluation of social programs should adhere to the principles of the impact evaluation process. The intervention project must be measurable for an impact evaluation to be done. Hence impact evaluation should determine if the intervention for social change can be assessed before a plan for the evaluation is set and executed. Secondly, the impact assessment should base on the current research about the program so that the results of the evaluation will act to build on the available research information on the social phenomenon and the intervening programs which aim at causing the require outcomes. Moreover, the use of various designs in the impact evaluation process must target specific groups even though the final impact should be assessed as either being positive or negative. According to Solorzano and Bank (2008, p.25), the measurements used in the assessment of a social intervention program or project must follow the standards set for a credible evaluation. Such standards include ethical considerations and appropriate treatment of the individuals participating in the evaluation process. The reporting of the results of an impact evaluation process should include both negative and positive effects of a social intervention program. This eliminates bias and thus leads to accurate evaluation for decision making and policy formulation. Asselin and Parkins (2009, p. 493) adds that the results of a good impact evaluation process must be shared for them to be valuable for policy implementation. All findings are thus availed to decision makers to be used for laying down policies which will promote the intended changes within the society. Thus in the evaluation of the Puntos de Encuentro program, the evaluation report should include the successes and failures of the program and both the positive and negative impact that the project through the communication programs caused within the communities in the Nicaragua region. Conclusion A good impact evaluation process should use proper methodologies in addition to following the standards of a credible evaluation. This will result to the evaluation of both negative and positive outcomes of a project intended to cause so that decision makers can implement suitable policies for bringing out social change. As explained in this essay a good impact evaluation process must be planned and thus measure the efficiency and effectiveness of a program in causing the needed change. Most importantly, the objectives of a social program are assessed by an impact evaluation process to ensure that accurate data is availed for decision making. References Asselin, J. & Parkins, J.R. 2009, "Comparative Case Study as Social Impact Assessment: Possibilities and Limitations for Anticipating Social Change in the Far North", Social Indicators Research, Vol. 94, no. 3, pp. 483-497 Bradshaw, S; Solorzano, I & Bank, A. 2006, “The Changing Nature of Change: A Nicaraguan Feminist Experience”, World Congress on Communication of Development, CI, FAO and World Bank. Baxter, G. 2004, "Impact Evaluation, Professional Practice, and Policy Making", New Library World, Vol. 105, no. 1, pp. 33-46. Deacon, B. 2007, “Global Social Policy and Governance”, Sage, pp. 93-104 Solorzano, I. & Bank, A. 2008, “Catalyzing Personal and Social Change around Gender, Sexuality, and HIV: Impact Evaluation of Puntos de Encuentro’s Communication Strategy in Nicaragua”, Horizons Final Report: Population Council, pp. 11-38 Weston, J. 1998, "Community Impact Evaluation", Journal of Environmental Planning and Management, Vol. 41, no. 6, pp. 781. Read More