Kirkpatricks Learning and Training Evaluation Theory - Case Study Example

Program evaluation case study The program evaluation model, selected for evaluating the program of proposed violence and sexual abuse prevention for youth to be undertaken in the schools is Kirkpatrick’s learning and training evaluation theory. This theory developed by Donald L Kirkpatrick in 1959 is arguably the wisest and effective choice, as this program to be evaluated is primarily based on the training the trainers module, for the evaluation of which, Kirkpatrick’s 4 level model is the best till date. As understood, effective program evaluation is the one that helps to gather and use information in order to learn continuously about the surveillance system and to improve its function and outcomes. In general any program evaluation model would encompass in it formative evaluation – that produces information that helps to form and refine the program and summative evaluation – that aims to delineate clearly the benefits generated by the program and the cost and essential conditions necessary to gain this conditions. Any program evaluation would follow a simple logic flow of input, activities, output, outcomes, impact, that to identify it as either promising or does not work program. Kirkpatrick’s training evaluation model was selected, as it helps to evaluate the project in 4 levels as reaction, learning, behavior and result. To evaluate the efficiency of the program that basically uses the variables of change in behavior and attitude among the selected groups, it requires a evaluation procedure that would concretely analyze the stages necessary to bring in these changes. In this given program where train the trainer’s module is the base, analysis of the subject’s reaction, learning and evaluation of the behavior as an measured output and results is very important. Hence it was felt that Kirkpatrick’s model would be better in the given scenario than any other proposed model. As an additional feature, the Jack Phillips addition of 5th level – return on investments can also be included – that would help to financially evaluate the benefit of the program. The next stage of development that weighs resource and activity against result and productivity would help us to concretely decide about outcome of the program. This model on the formative evaluation side would help to generate the parameters and research methodology types, required to generate the necessary data. On the summative evaluation side this model would give out the effective cost involvement and benefit. The next step after ascertaining the model would be to assess the salient process variables. As known, variables may fall into 3 categories as independent, dependent and control. The variables in this program will fall under control variables – background, classificatory, or organismic. The variables as age, sex, IQ, socio economic status, educational level, motivational level are to be controlled, as these factors may lead to any unexpected bias on the outcomes of the program. So it becomes very essential to consider these variables while planning. Also intervening variables as learning, intelligence, perception, motivation, need, self, personality, trait and feeling are also to be considered. The program if designed as a quasi – experimental program, where the design includes a minimum and random assignment of study group and control group, the comparison of which would effectively portray the results. In this program the level of attrition should be taken into account, as it is a very important variable that could affect the outcome. In general the variables that need to be assessed are the behavior, attitude, efficacy and knowledge. As the variables are fixed, the next stage would be planning the operationalization of the variables. This means, a process of describing variables in concrete terms, which would facilitate is measurement. All the variables would be assessed as a statistical difference seen between the control and a study group, when compared on each year of the progression of the study. The knowledge of content that the teachers, children and parents would have acquired could be judged efficiently with the help of the questionnaires and post program tests that would bring out the amount of knowledge that would have been ingrained in the group. Behavior – behavior can be assessed through self reported findings, reported incidences and the questionnaires. Efficacy on the other hand could be brought out by questionnaires or by narration of the confidence and emotions over the skill acquired and by pretest and posttest that would show their significant confidence difference. Attitudes – attitudes again could be brought out by properly framed questioners. For all the factors time efficiency can also be taken as a judging tool. The research methodology would begin with an informed concent obtained from the students and the parents to participate in the research. Then half of the population with appropriate general criterions would be selected as sample group and be given with intervention school curriculum program and other half would remain as the control group, whose attitudes, behaviors, knowledge etc would serve as a baseline for prevention programs. The design of the project would be quasi - experimental. The various methodological tools that would be used in the study could be pretest and post test assessment using well framed questioner that could bring out the amount of knowledge gained, self reported findings, reported incidence of violence and abuse from the school registers, interview with the subject group along with the narration of the confidence obtained over the information and skill acquired. The next stage would be data analysis that would determine the effectiveness of the project. Statistical significance, based on the level of confidence would help to conclude the difference between the groups to derive at the outcome as whether the results from the treatment are delivered or not. A probability value of .05 would be widely accepted as threshold of statistical significance and a probability below would indicates that a difference of this magnitude would happen only by chance. A multiple linear regression can be followed in quantitative data, whereas in qualitative data detective and inductive coding categorize would provide information. As a feedback it would be suggested that the program outlined with a large sample size and in a background with multiple risk factors, scope of population to be reached, multi dimension preventive nature of approaches to be implemented in a critical area of need backed up with unlimited funding would achieve the proposed outcome. The trainer’s training model though effective has high shortcomings of increased variability’s in training capacity and in taking capacity, that would vary with individuals. The proposed long term and short terms might be achieved but the sustainability and replicability of the results obtained is highly under doubt as it is quite natural that as the effect of intervention fates away the effects also would gradually wane off. A less widely accepted but very important standard for demonstrating the effectiveness is the long-term sustainability of the expected and achieved outcomes. This is particularly very important in evaluating the violence prevention program as the beneficial effect could quickly diminish after the youth leaves the treatment setting or program to return to their usual environment. Effective program should produce long-term changes in individual competency, environmental conditions and patterns of behavior. On practical level program could be considered to have demonstratable sustainability if the effects of intervention continue for at least a year after the treatment. Bibliography: 1. Kirkpatrick, Donald (1994). Evaluating Training Programs. San Francisco: Berrett-Koehler Publishers, Inc. Read More