Practical Program Evaluation - Case Study Example

Questions: 1. What would be the purpose of the evaluation? Identify intended users and intended uses of evaluation The purpose of the intended evaluation is to systematically collect data that seeks to answer questions on ways of improving the way the intended program will be delivered. It would be noted that Homes Against Homelessness is currently seeking to feed its philanthropic donors on a program that has taken place. This means that the purpose of the evaluation shall also be a platform to reflect on the outcome and effect that the program created. As a nonprofit organization, evaluation of programs of Home Against Homelessness’s is a very crucial part of the organization’s day to day activities so that donors can gain credibility with the organization that the contributions they are making to the organization are being used for the right purposes and enhance the present evaluation. As noted by Wholey (2010), primary intended users are part of stakeholders and refer to those people who will work directly or indirectly with the evaluator in the course of evaluating the program. This means that within Home Against Homelessness, a number of key stakeholders shall be identified and given the task of intended users so that in a collaborative manner, the evaluator shall achieve the purpose and goals of the evaluation with them. For the sake of credibility, intended users shall be selected from both within and without the organization. From within the organization, intended users shall include events manager, outsourcing director, operations manager, and the finance manager. From without the company, there shall be independent auditors and resource consultants included to enhance the neutrality of the entire evaluation process. After the entire evaluation is complete, it is expected to have several uses and play very important roles in the day-to-day running of Home Against Homelessness. The very first use of the evaluation is a documented proof of the efficiency of its programs, of which it uses external funding and donations. Often times when donors give towards charitable courses such as the ones that the organization is pursuing, they would want a guarantee that the contributions they made was used for the intended purposes and if so, at what cost (Williams, 2005). Due to this, the evaluation documentation shall be a reference material for any such referrals to be done on the effectiveness of the organization’s program. What is more, the evaluation shall be used as a critical yardstick for measuring future successes and failures with programs. That is, the level of success or failure of the program being evaluated will indicate to the stakeholders, things to repeat and those to avoid in all future programs. 2. What type of evaluation approach would you recommend? The evaluation shall be approached from a more summative perspective as against formative perspective. The reason for summative approach is in the kind of aims that the evaluator wants to achieve by undertaking the current evaluation. Generally, it would be noted that summative evaluation gives the evaluator the opportunity to measure program outcomes rather than the program delivery itself. This means that with a summative approach, attention will be given to the aims and objectives of the program, determined at the planning stage of the program s that it will be easier to raise the question of whether or not the program achieved these aims and objectives. This is said because the program outcomes and impacts which a summative evaluation seeks to measure are the direct reflections of the goals and objectives that were intended for the program. A major advantage for using a summative approach in this context is that the background and nature of Home Against Homelessness as a nonprofit organization requires that the organization gets a rapid evaluation system that can give stakeholders an indication as to whether or not their contributions are making any meaningful impacts in society. By the close of such summative evaluation therefore, it should be possible for any philanthropist who has made contributions to the organization to have a quantitative measure of the level of impact that he or she has made to society through the eradication of homelessness. 3. What would you measure? Write three researchable questions. As part of the summative approach, which looks more into the outcome and impact of the program that is undertaken, the evaluation shall be seeking to measure the changes that Home Against Homelessness has brought to society through its program. That is, the value of the program in changing the fortunes of the ordinary person in society who sees Home Against Homelessness as hope for the future shall be determined or measured. What is more, and related, the level of useful implementation of funds made available by philanthropists shall also be measured as part of the evaluation. To achieve the goals on what to measure as a result of the evaluation, it will be important to have researchable questions that the entire evaluation process shall be seeking to answer. In effect, these three questions shall be the fulcrum around which the conduct of the evaluation shall revolve: 1. Which management decisions about the program can be influenced by the results produced from the evaluation? 2. What accounts for the type of outcomes resulting from the evaluation of the organization’s program? 3. What are the specific differences that the program brings to intended beneficiaries and society in general? Once these questions are answered through various forms of data collection methods, it is expected that the intended parameters to be measured in the evaluation shall successfully be measured. 4. What type of evaluation method(s) might be appropriate? Discuss the pros and cons of the evaluation method(s) you are suggesting. Generally, Wholey (2010) notes that the summative approach that has been intended for use for the evaluation is more appropriate when used with a qualitative evaluation method. However, measuring the impact that a quantitative method would also have on the study when combined with the qualitative method, the evaluator shall settle for the use of a mixed method. That is to say that there shall be the combination of qualitative and quantitative methods. In the qualitative method, an open ended approach shall be taken to data collection whereby the evaluator shall be limited to the use of descriptive data and facts about the program. Even when it comes to the actual data collection exercise, which shall involve the collection of data from respondents, a more open ended instrument such as an interview shall be used. For the quantitative method however, a more structured and numeric approach shall be taken towards the data collection and entire evaluation procedure. It is worth noting however that whereas the qualitative method shall be very dominant with the data collection stage of the evaluation, the quantitative method shall dominate the data analysis phase of the evaluation. Once the mixed method is used, there are some key advantages that the evaluator is expected to enjoy. The commonest of these is that combining two methods will create room for the weaknesses of one of the methods to be absorbed into the strengths of the other. That is in situations or cases where the qualitative method cannot stand as an independent method to make the evaluation successful; the researcher will quickly fall unto the use of the quantitative method to achieve the same purpose for which the qualitative method was to achieve. What is more, the nature of the present evaluation is expected to involve a very wide range of respondents and participants when it comes to collecting data for decision making. To this effect, it is important to use such an evaluation method that can easily cater for individual differences. Meanwhile, the mixed method has been identified in literature to serve this purpose more easily (Williams, 2005). For example when it comes to the use of qualitative methods such as interview, it may not be generally appropriate for respondents with very busy schedules who cannot sit to answer questions at a go. In such cases, the use of quantitative methods such as questionnaire becomes appropriate so that in comfort of their spare time, the respondents can attend to the questions. The merits of the mixed method notwithstanding, there are some key weaknesses or shortfalls that may also be associated with it. A typical example is that using the mixed method is generally time-consuming. That is, the evaluator is required to prepare separate evaluation methods for both the qualitative and quantitative methods when single methods like qualitative alone or quantitative alone would have provided that only a single data collection instrument is prepared. Meanwhile, undertaking professional evaluation of this nature is often time bound. In cases where the evaluator wants to meet deadline at all cost, the temptation is that effective evaluation process will be compromised for promptness of the evaluation. 5. Who are the stakeholders/customers and how and when might stakeholders be most fruitfully engaged? In the words of Wholey (2010, p. 28), “evaluation stakeholders are the individuals, groups, or organizations that can affect or are affected by an evaluation process or its findings, or both.” What this means is that all people from whom the researcher shall collect data shall be counted as stakeholders for the evaluation. This is because the responses that shall be collected from these people will go a long way to affect the evaluation findings. The responses that will be collected from these people shall also go a very long way to determine the conclusions that will be drawn on the program. With this noted, four major types of stakeholders shall be used. The first are staff and workers of Home Against Homelessness, who shall be fruitful in collecting specific quantitative data that has to do with facts and figures about the program and the organization as a whole. The second group shall be intended beneficiaries, from whom qualitative data shall be collected on the impact that the program has brought them as people. There shall also be philanthropists who shall give some quantitative and qualitative data on their contributions to the program and what they expected out of the program. Finally, expert resource persons shall be included to give professional assessment to the program based on parameters of measurements given to them. Reference List Wholey Handbook of Practical Program Evaluation. 3rd Edition. John Wiley & Sons, Inc. Williams, M. (2005). Leadership theories, role models—and common sense. In Leadership for leaders (pp. 29–47). London, UK: Thorogood. Read More