Evaluation Models in Organizations - Report Example

EVALUATION MODELS IN ORGANIZATIONS al Affiliation Introduction Evaluation refers to a systematic process that determinesthe effectiveness of a process, a program or an approach. Evaluation approaches aid in determining the efficiency of instructional involvements. (Schmalenbach, 2005) urged that training, being an investment in the organization, it is vital for the organization to know whether their investment has been efficiently spent or otherwise. Additionally, the expenses and costs that an organization has to meet have necessitated the significance of evaluating the process to determine whether the benefits that accrue an organization outweigh the costs. This paper critically assesses the different evaluation approaches of training by comparing and contrasting each one of them. It will provide an insight on which models or approaches seem similar or compatible. Further examinations will be carried out to know the ways they are similar or compatible and the differences in orientations, and how the differences will manifest in an evaluation. Moreover, the evaluation is aimed at revealing which of the models or approaches would fit within the context of any organization. Body The models to evaluating training include The Kirkpatrick Model (TKM), CIPP Evaluation model, CIRO approach & Input, Process, Output, Outcome (IPO) Model (Bushnell, 1990). The Kirkpatrick Model was introduced in 1959 is the most common model of evaluating training and development programs. The TKM is based on four levels of evaluation. These levels are reaction, learning, behavior and results (Phillips, 2010). However, the model has experienced an addition of a fifth level, incorporating the idea of measuring the return on investment (ROI). The table below compares and contrasts the different models to evaluation. System-based include CIPP, TVS and IPO. While Kirkpatrick is goal based Kirkpatrick (1959) CIPP Model (1987) TVS Model (1994) IPO Model (1990) 1. Reaction: to collect information on trainee reactions at the conclusion of a training program (Praslova, 2010)  1.Context: getting data about the situation to make a decision on educational needs and to institute training program goals or agenda  1.Situation: gathering pre-training data to ascertain current levels of performance within the organization and defining a desirable level of future performance 1. Input: evaluation of indicators of system performance like participants qualifications, accessibility of materials, appropriateness of training, etc. 2. Learning: to assess whether the learning intentions for the program are achieved 2. Input: recognizing educational strategies most probable to accomplish the preferred result 2. Intervention: recognizing the basis for the existence of the niche between the current and enviable performance to discover if training is the answer to the problem 2. Process: adopts scheduling, design, progress, and delivery of instructional programs 3. Behavior: assessing if job performance changes because of training 3.Process: evaluating the implementation of the educational program 3. Impact: evaluating difference between the pre- & post-training data 3.Output: Gathering data resulting from the training interventions 4. Results: to review costs vs. payback of instruction alprograms, i.e., organizational effect in terms of reduced expenditure, better quality of work, augmented capacity of work, etc. 4. Product: collecting information concerning the results of the learning intervention to construe its value and merit 4. Value: determining disparity in quality, output, service, or sales, expressed in monetary terms (Stufflebeam, 2001). 4. Outcomes: long-term outcomes related to development in the organization’s profitability and competitiveness Based on the comparisons depicted in the table above, the CIPP Model, TVS and IPO models seem compatible and similar. These models define the steps needed to accomplish the intentions of the training. For example the CIPP model aids the instructor in recognizing the best strategies for education that will facilitate the training to realize the desired outcomes of the training (Stufflebeam, 2007). Similarly, the TVS model (1994) has its number two process having the objective of determining if the training program will be the solution to the problem that exist prior to training. The IPO model in the same regard has the second process engage the planning concept. All these models therefore are aimed at recognizing the best strategies to institute during the training program. CIPP, TVS and IPO also are also based on clear evaluation methodologies that engross qualitative, quantitative method or a hybrid of both methodologies. Therefore it is easy for practitioners to determine the effectiveness of the program through quantitative measures such as profitability, as in the case of the IPO model (Dahiya and Jha, 2011). In the TVS model, the value of the training program could also be evaluated by measuring sales. Likewise to the CIPP, the end result of the training program can be valued by gathering information concerning the results of the training program and interpreting the worth. Models or Approaches with Different Orientations and How the Differences Manifest In an Evaluation The Kirkpatrick differ, greatly with the other three models~CIPP model, TVS model and IPO model). To begin with, Kirkpatrick’s model is goal- based (Praslova, 2010). This assists the various instructors of the training program reflect on the intention of evaluation varying from entirely technical to stealthily political intention. Yet, Kirkpatrick model does not identify the steps required to accomplish purposes and does not attend to the ways to make use of the results to improve training. The complexity for practitioners using Kirkpatrick approach is in choosing and employing suitable evaluation methodologies that engross qualitative and quantitative methods (Wholey, et al, 2010). As a result of the obvious simplicity, instructors quickly choose the Kirkpatrick models or the goal based models without carefully evaluating their particular needs and organizational spending power or establishing how they’ll affect the model and the outcome (Reiser and Dempsey, 2011). Unsurprisingly, several firms do not apply the whole model, and training results to being evaluated merely at the reaction level, or if further, at the learning stage. As evaluation levels goes higher, the complications drawn in amplify. This explains the reason why organizations usually apply reaction and learning levels only. On the contrary, systems-based models such as CIPP, TVS and IPO appear to be more valuable with respect to thinking about the general circumstance and situation but they may not provide adequate course. (Holmboe, et al, 2006) echoed that systems-based models may not characterize the forceful connections between the design and the assessment of training. Few of the models or the approaches offer detailed illustrations of the processes engrossed in all the steps. None of the models present mechanisms for evaluation. In addition, these models do not deal with the mutual process of evaluation, that is, the diverse roles and tasks that people may play during an evaluation process. Models or Approaches Fit Within a Context an Organization or Organizations I Work For The best model that would best fit the organization that I work in would be the Kirkpatrick’s model that incorporates the fifth level (ROI). The reasons for this are based on the various stakeholders in the company. (Nickols, 2005) illustrate that these stakeholders include the trainers, the training managers, the funding managers and the trainees or employees. Admittedly, the trainees are inclined to care principally about two aspects: application of the subject matter, and the degree to which the program utilizes their time. Training merchants ought to find out whether their customer, the training section, is contented with the training they acquired. Stakeholders want to know what the participants feel. Incorporating ROI by determining whether the returns of the investment in training is positive will enable the financiers of the program establish whether to execute the training program consistently or abandon it (Barnett and Mattox, 2010). Discussion Deciding on Models & Approaches to Use for Evaluation The questions the evaluation intends to answer determine when and which models and approaches to use in evaluation. Such questions include what happens if a new approach is used? Will the program work in a different context? (Arthur, et al 2003) argued that it also depends much on the information gathered at the onset of training. On how to choose a model or evaluation approach, the factors that influence the decision include the nature of the question the program is intending to address, the challenges of the approach or model face and the possibility of employing a specific model to the training program or circumstance. It’s equally crucial to establish what the structure of the training program will sustain, what trainees will assent to, and what the organizational resources and time limitations entail. Criteria for Determining Most Ideal Model and Approach for Evaluation Context The criteria is chiefly based on the factors to consider in determining the best appropriate model and approach for a given evaluation context matter. (Rossi, et al, 2003) reckoned that such factors include evaluation purpose, the program features, the information required and what’s going to be done to the results of evaluation. Availability of resources such as time, money and human resources is another criterion to determine the most appropriate approach and model. Discussion on Implications of Utilizing Different Evaluation Models The implication of using system-based approaches is that the person evaluating examines and measures definite practices and outcomes; and interviews the trainees. This averts tunnel vision, or only observing the program as it relates to the anticipated goals, thereby overlooking many positive or harmful involuntary side-effects. The implication of using goal-based approaches is that the evaluator will mainly concern himself with achieving the goals set by the program (Fitzpatrick, Sanders and Worthen, 2004). This hampers caution as other areas pertaining program not be addressed. In conclusion, the evaluation models that organizations adopt are usually goal –based. However, it is recommended for organization to adopt a hybrid-type, incorporating system-based approaches. This will ensure effective evaluation of programs. Besides, it is important for organizations to recognize the need to use ROI evaluating the effectiveness of any program, whilst adhering to the need to address qualitative measures. References Arthur Jr, W., Bennett Jr, W., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: a meta-analysis of design and evaluation features.Journal of Applied psychology, 88(2), 234. Barnett, K., & Mattox, J. R. (2010). Measuring Success and ROI in Corporate Training. Journal of Asynchronous Learning Networks, 14(2), 28-44. Dahiya, S., & Jha, A. (2011). Review of training evaluation. International Journal of Computer Science and Communication, 2(1), 11-16. Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines. Holmboe, E. S., Rodak, W., Mills, G., McFarlane, M. J., & Schultz, H. J. (2006). Outcomes-based evaluation in resident education: creating systems and structured portfolios. The American journal of medicine, 119(8), 708-714. Nickols, F. W. (2005). Why a stakeholder approach to evaluating training.Advances in Developing Human Resources, 7(1), 121-134. Phillips, P. P. (Ed.). (2010). ASTD Handbook for Measuring and Evaluating Training. American Society for Training and Development. Praslova, L. (2010). Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. Educational Assessment, Evaluation and Accountability, 22(3), 215-225. Reiser, R. A., & Dempsey, J. V. (2011). Trends and issues in instructional design and technology. Pearson Merrill Prentice Hall. Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2003). Evaluation: A systematic approach. Sage publications. Schmalenbach, M. (2005) Training evaluation–what’s it all about?. Stufflebeam, D. (2001). Evaluation models. New directions for evaluation,2001(89), 7-98. Stufflebeam, D. L. (2007). CIPP evaluation model checklist. Western Michigan University. The Evaluation Centre. Retrieved June, 2, 2009. Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (2010). Handbook of practical program evaluation (Vol. 19). John Wiley & Sons. Read More