Evaluation Types, Approaches, Processes and Frameworks - Research Paper Example

? Evaluation Types, Approaches, Processes and Frameworks. The reasons to gain more knowledge about evolution theory are beyond the tactic or why distinction. Evaluation theory goes further than helping one make clear judgments about what sort of methods to use, in what circumstances to use them, and toward which forms of evaluation influence. This article goes ahead to discuss the importance, types, approaches and frameworks of evaluation. Introduction Evaluation is definitely an integral part of many instructional models of design. Evaluation methodologies and tools assist determine how effective the instructional interventions are. Evaluation is characterized as either formative or summative . Broadly , formative evaluation focuses on what leads to an intervention working, and summative evaluation, on the other hand, looks at specifically the short-term to long-term outcomes of an intervention on the groups targeted. Formative evaluation takes place before the project, and also during the project so as to improve the design of the project as it is being put into implementation. Formative evaluation is usually used in qualitative ways of inquiry. Summative evaluation is used during and after the project implementation, and it is associated with more decision, quantitative methods. The difference between formative and summative evaluation is blurred. Generally it is of paramount importance to have knowledge of how an intervention works, and also if it worked. It is thus important to capture and make assessment of both qualitative and quantitative data. Importance of Evaluation The first importance of evaluation is that evaluation theories are a way of consolidating lessons learned, that is, of synthesizing prior experience. They help evaluators develop a more sophisticated and nuanced understanding of the way organizations make decisions and may be influenced by evaluation findings. Theories enable us to learn from the experience of others. Second, comparing evaluation theories are a useful way of identifying and better understanding the key areas of debate within the field. Comparative study of evaluation theory likewise helps crystallize what the unsettled issues are in practice. A third reason for studying evaluation theory is that the theory should be an important part of our identities as evaluators, both individually and collectively. If we think of ourselves in terms of our methodological skills, what is it that differentiates us from many other people with equal (or even superior) methodological expertise? Evaluation theory. But people come for evaluation through quite varied pathways, many of which don’t involve explicit training in evaluation. That there are myriad pathways into evaluation is, of course, a source of great strength of the field, bringing a diversity of skills, opinions, knowledge sets, and so on (Schwandt 2002).. Types of evaluations Process Evaluation Process evaluations help stakeholders see how a program outcome or impact was achieved. Overemphasizing outcome evaluation of the cost of other types, especially process evaluation, is a disservice to nonprofits and the sector. This is because the process evaluation allows a nonprofit to look at how it develops itself, its structures, its supporting programs like communications and marketing, even fund development in order to get to the outcomes everyone wants it to achieve. In other words, process evaluations document the process of a program's implementation. Process evaluations help stakeholders see how a program outcome or impact was achieved. The focus of a process evaluation is on the types and quantities of services delivered, the beneficiaries of those services, the resources used to deliver the services, the practical problems encountered, and the ways it was handled. Taking process evaluation a step further, it can also look at the processes of program, management and infrastructure together that is the capacity of an organization to deliver on its promised outcomes. Approaches to Evaluation Commonly used approaches to educational evaluation have their roots in systematic approaches to the design of training. They are typified by the instructional system development (ISD) methodologies, which emerged in the USA in the 1950s and 1960s and are represented in the works of Gagne and Briggs (1974), Goldstein (1993), and Mager (1962). Evaluation is traditionally represented as the final stage in a systematic approach with the purpose being to improve interventions or make a judgment about worth and effectiveness. More recent ISD models incorporate evaluation throughout the process. Six general approaches to educational evaluation can be identified as follows: Goal-based evaluation Goal-free evaluation Responsive evaluation Systems evaluation Professional review Quasi-legal Goal-based and systems-based approaches are predominantly used in the evaluation of training. Goals-based evaluation Evaluation has typically involved measuring whether predetermined targets have been met. Many people are familiar with the term ‘SMART’ (Specific, Measurable, Attainable, Relevant, and Timely) targets. These types of targets fall under the banner of goals-based evaluation. Goals-based evaluations have objectively-set targets usually determined by people responsible for the funding or implementation of the project.  Goals-based evaluation does not question whether the selected goals are valid, nor whether appropriate measures of effectiveness are being assessed. Recent behavior change and research and evaluation practice propose the use of goals-free evaluation. Goals-free evaluation Goals-free evaluation looks at emerging and the unintended consequences of a project, by looking at the actual effects without pre-empting what these may be. This type of evaluation focuses on the change process within the target group. The goals-free evaluation parallels a social learning approach to evaluation which can be desirable where the goal of a project seeks to build capacity of people to manage a complex issue. In such cases, relying solely on a goal-based evaluation may not be appropriate as it can be difficult to set targets and quantify a process of change, as well as capture change which one may not anticipate (Bramham 1994). A key point to consider in setting goals and indicators of success is the validity of the indicator. This is especially important when you are looking to measure a largely “unobservable” behavior (for example, those that happen within the home, and cannot be readily objectively observed). In such cases, proxy indicators are often used (for example, household electricity use) but these indicators may be gross measures that consist of many varied behaviors, of which only one may be of interest. The question is how to establish meaningful indicators? Frameworks for Evaluation Goal-based and systems-based approaches are predominantly used in the evaluation of training. Various frameworks for evaluation of training programs have been proposed under the influence of these two approaches. The most influential framework has come from Kirkpatrick. Kirkpatrick’s work generated a great deal of subsequent work. Kirkpatrick’s model (1959) follows the goal-based evaluation approach and is based on four simple questions that translate into four levels of evaluation. These four levels are widely known as reactive, learning, behavior, and results. On the other hand, under the systems approach, the most influential models include: Context, Input, Process, Product (CIPP) Model; Training Validation System (TVS) Approach; and Input, Process, Output, Outcome (IPO) Model. On the other hand, systems-based models (e.g., CIPP, IPO, and TVS) seem to be more useful in terms of thinking about the overall context and situation but they may not provide sufficient granularity. Systems-based models may not represent the dynamic interactions between the design and the evaluation of training. Few of these models provide detailed descriptions of the processes involved in each step. None provide tools for evaluation. Furthermore, these models do not address the collaborative process of evaluation, that is, the different roles and responsibilities that people may play during an evaluation process. Conclusion Different approaches to evaluation of training discussed herein indicate that the activities involved in the evaluation of training are complex and not always well-structured. Since evaluation activities in training situations involve multiple goals associated with multiple levels, evaluation should perhaps be viewed as a collaborative activity between training designers, training managers, trainers, floor managers, and possibly others. There is a need for a unifying model for evolution theory, research, and practice that will account for the collaborative nature of and complexities involved in the evaluation of training. None of the available models for training evaluation seem to account for these two aspects of evaluation. Existing models fall short in comprehensiveness and they fail to provide tools that guide organizations in their evaluation systems and procedures. Not surprisingly, organizations are experiencing problems with respect to developing consistent evaluation approaches. Only a small percentage of organizations succeed in establishing a sound evaluation process that feeds back into the training design process. Evaluation activities are limited to reaction sheets and student testing without proper revision of training materials based on evaluation results. Perhaps lack of experience in evaluation is one of the reasons for not consistently evaluating. In this case, the organization may consider hiring an external evaluator, but that will be costly and time consuming. Considering the need for the use of internal resources and personnel in organizations, expert system technology can be useful in providing expert support and guidance and increase the power and efficiency of evaluation. Such expert systems can be used by external evaluators as well. Strong, completely automated systems offer apparent advantages, but their development and dissemination lag behind their conceptualization. Future research needs to focus on the barriers to evaluation of training, how training is being evaluated and integrated with the training design, how the collaborative process of evaluation is being managed and how they may be assisted. This will be helpful in guiding the efforts for both the unifying theory of evaluation and in developing automated evaluation systems. Possible explanations for inadequate evaluations include: insufficient budget allocated; insufficient time allocated; lack of expertise; blind trust in training solutions; or lack of methods and tools References DePoy, E., & Gilson, S. F. (2008). Evaluation practice: how to do good evaluation research in work settings. New York: Routledge. Schwandt, T. A. (2002). Evaluation practice reconsidered. New York: Peter Lang. Human resource management review. (1991). Greenwich, Conn.: JAI Press. Bramham, J. (1994). Human resource planning (2nd ed.). London: Institute of Personnel and Development. Read More