The Use of Evaluation and Assessment in Aviation Training - Essay Example

? Assessment and Evaluation in aviation learning It is indispensable that the evolution of the world has been accompanied with various technological developments and concepts to the extent that the world can no longer do without them. One of the pivotal areas where this is manifested pertains to the applicability of ISD model. There has been the increasing need to adopt new technologies for identification practice, triggered by the increased proliferation of the information technologies, electronic transactions, service delivery and data processing. Needless to say, the assessment and evaluation as components of ISD model are gradually and persistently emerging to be the main components of identification used in aviation training. It is widely cited that advancements in technologies have been accompanied by wider research and application of education programs. The claim is a connotation that emphasizes how the effectiveness of these education research models may have gone beyond reproach. This paper carries out a critical evaluation of the claim and its implications on the development of aviation training. It explores the applicability of the ISD model components, focusing on assessment and evaluation as applied in aviation training. While cross examining the use of evaluation and assessment in aviation training, it will be of significant importance to understand ISD and its applicability in the learning process, as well as how its components are interlinked to yield a well structured model. An Instruction design model is a process traditionally utilized by developers of instructions and designers trainers to represent flexible and dynamic guidelines necessary for an effective performance and training for support tools. The ISD model is composed of five phases (Rothwell & Kazanas, 2008). These include design, analysis, implementation, development, and evaluation. Analysis Analysis is the phase in which classification of the instructional problem is implemented. Additionally, it entails the establishment of objectives and goals, as well as identification of the learners existing skills and environment. Design Design phase specifically deals with objectives of the learning process, content, exercise, lesson planning, objectives and assessment criteria, as well as media selection. This phase should be specific and systematic. Development It is the phase in which developers and instructional designers assemble and create content assets blueprinted. In addition, graphics, content and storyboard are made in this phase. Programmers are involved in developing and integrating technology for the learners. Implementation At this stage, procedure used in facilitator’s training is developed. The training must cover the learning outcomes, testing procedure and method of delivery, as well as Evaluation. Evaluation phase has two parts: summative and formative. The Formative evaluation processes are presented in each ADDIE stages. Summative evaluation, on the other hand, consists of tests that are designed for specific criterion-related item, and provides feedback opportunities identified by the users. Most instruction design models that are currently in use spanned off from ADDIE instruction design model (Rothwell & Kazanas, 2008). These include the dick and Carey and Kemp instructional system design models. The use of photo typing is one such improvement of the model. It involves receiving formative or continual feedback while creating the instructional materials. The model helps in saving time and money i.e. capturing problems when not yet fixed. The figure below shows the ADDIE model. The Jerold Kemp instructional design model and method defines nine components of the instructional design while adopting a continuous evaluation or implementation model (Jerrold, 1977). Jerold Kemp takes a wider view. This model adopts an oval shape that conveys the development and design of continuous cycles that require constant designing, assessment and development. This ensures that the instructions are well designed. This model is nonlinear and systematic, and encourages designers to attend to all areas appropriately. Kemp model is applied in the development of instructional programs of blending pedagogy, technology and content. This ensures the content is reliable and effectively delivered for efficient learning to take place. The nine main elements identified by Kemp are identifying instructional problems, specifying goals of designing instructional programs, examining the characteristic of the learner that should get the attention during planning, identifying subject contents, analyzing task components related to goals that are stated, stating the learners instructional objectives, sequential content in each instructional unit for the purposes of logical learning, Designing instructional strategy for each learner to master the objectives, planning the instructional messages and the delivery of the message, developing evaluation instruments for assessment of the objectives, selecting the resources that support learning and instruction (Gary & Steven, 2010). Figure 2 below shows the Kemp instructional design model. The 9 Events Gagne Instructions (Gary & Steven, 2010) Robert Gagne is known to one of the foremost contributors of the systematic approach to instructional design and training (Jerrold, 1977). The followers of Gagne and Gagne himself are known as behaviorists. This is because their focus is on behavior that results from training, His book the conditions of Learning’ did identify mental learning conditions. He came up with a process of nine steps known as events of instruction. These processes correlate and address the learning conditions (Rothwell & Kazanas, 2008). The nine steps are; gaining of attention, informing the learner of the objectives, stimulating recalling of prior learning, presenting stimulus materials, providing guidance to the learner, eliciting performance, providing feedback, assessing performance, and enhancing retention transfer. Bloom's Learning Taxonomy Bloom , alongside other educational psychologists, developed classification criteria of levels of behavior in the intellectuals as. He found that over 95 percent of the question tests that the students encounter needed thinking and the recalling of information, only at the lowest level possible. In his study, Bloom found six levels in the cognitive domain (Gary & Steven,2010). This was from the basic facts recognition, being the lowest level, increasingly through more abstract and complex mental levels, to the evaluation level, which is the highest order. Examples of Verb representing intellectual activity are on each level as listed in fig 3. Figure 3 Kemp Instructional Design Model (Gary & Steven,2010) Knowledge: define, duplicate, label, arrange, memorize, list, order, name, recognize. Recall, relate, reproduce, and repeat state. Comprehension: discuss, classify, explain, describe, express, indicate, recognize, locate, report, review, translate, select, identify, restate Application: choose, apply, dramatize, demonstrate, employ, interpret, illustrate, practice, operate, sketch, use, schedule, solve, write. Analysis: analyze, calculate, appraise, compare, categories, contrast differentiate, criticize, distinguish, discriminate, question, examine, experiment, and test. Synthesis: arrange, collect, assemble, compose, create, construct, design, formulate, develop, manage, plan, organize, prepare, set up, propose, and write. Evaluation: Appraise asses, argue, attach, compare, choose, defend, estimate, predict, judge, rate, select, core, support, value, and evaluate. Donald Kirkpatrick published his ideas in a series of article in the American Journal of society of training Directors. These articles were included in Kirkpatrick's book’s Evaluating Training Programs (originally published in 1994; now in its 3rd edition book Evaluating Training Program (Jerrold, 1977). The theory is increasingly becoming the most popular and in training, evaluation and learning. His four level models is an industrial standard across the training groups and the HR. Kirkpatricks four-level evaluation model are: REACTIONS-What they felt and thought about the training LEARNING- The resulting increased capability or knowledge. TRANSFER- Extents of behavior and improvement in capability and implementation RESULTS- The impact of the environment or business that resulting from the performance of the trainee. All the measures are recommended for meaningful and evaluation of learning in an organization, although their applications broadly increase in complexity, and cost, through the levels from level 1-4 as illustrated in the figure below. Fig 4; Kirkpatrick's 4 Levels of Training Evaluation The use of evaluation and assessment in aviation is a demonstration of how the ISD model is applicable in the real life situation. Aviation training can be categorized into four generations. These are simulation, apprenticeship, customized training, and safety. Each training generation enriches the subsequent generation (Tony, 2007). Since 17th December 1903 when Wright brothers established the controlled, powered flight, they laid a foundation for training in aviation. This is still used in training of pilots in modern day organizations. The structure of the training according to this model included the classroom trainings or the ground school, and the in aircraft trainings. Pilots known give experience guides the novice pilots. They do so by demonstrating the skill and by transmitting knowledge, until the students can attain the competence level. The simulation is the second generation. In 1929, the trainer was invented and this lead to simulation generation. This meant that flight schools were not to depend on conditions of weather for flight training to take place. Simulators are now incorporated in flight training. In this simulation stage, the apprentice model was still being used within- aircraft training and the classroom slightly supplemented by the simulator trainings. Mastering of performance standards of the flight was still the core objective of the training. The focus was to improve on the aircraft components and aircraft systems. Safety of 1979 to present is the third generation. This aviation training generation was started in 1979 during the NASA conference (Gary. & Steven, 2010). The airline representatives converged at this conference for a discussion about the increased number of accidents in aviation industries occurring due to human errors. This led to the training of Crew Resource Management. This involves the effective usage of resources in order to achieve efficient and safe operations. This training differed from the ground instructions that exclusively focused on the system of aircraft and air knowledge. However, there has been a considerable positive gain in attitudes, and the yearly training in CRM is mandatory for most of the military pilots and airlines. The apprentice model is still being used in training with in air trainings or simulator. This aviation training generation is a reflection of the current training in aviation industries. The customized training is the fourth and last generation. It is the most current training system that was designed to sort learners rather than for learners. The students are evaluated and sorted into grades as A, B, C or fail/pass. Standardizing instruction gives allowance for the company simple and valid methods of comparing students. Competency based training is an example of an aviation training program that has undergone the four transformation generations. It applies assessment and evaluation as training tools for aviation experts. Competency training is whereby students are trained and assessed to make them attain specified standards that define knowledge, skill and behavior needed to effectively and safely accomplish a certain task. Competency-based training involves an ray of tasks. This includes; accomplishing an individual need of training, ensuring the training routine meets the standards that are specified, and reflecting on what one should do at the work place or in life. Before the assessment is done, students are taught structured training, addressing the standards needed to get the qualification such as an endorsement or licensing. All training has to be planned accordingly. A training plan sets up all knowledge, behavior and skills to be taught in a logical sequences of the events of the training, how the training should be done and recorded, flight training and coordination theory, as well as when and how has to be conducted. It has to give room for sound adjustments and allow for planning in case operation restrictions and unexpected weather arises. The plan is made accessible to the staff. The Chief Flying Officer monitors and controls the training. Students are assessed according to a standard measure. In order to ensure certain quality of assessment, the standards have to be valid, measurable, authentic, objective, current and sufficient. The approach combines understanding, technical skills, knowledge and applications in assessment. There is much emphasis on assessment than on training. Competence standards are tools that are applied to test an individual’s competency to accomplish a given task. Flight measures promoted by CASA differ slightly from National Industry measures controlled by Education Department. This measures compliment aviation flights training and accommodate flight training for the instructors, as well. The tools that are used for assessment in CASA competency based trainings help simplify tasks. These include training records, achievement records, flight standards, examination results, pilot log book, instructions for candidates and assessors, CASA test forms and approve testing manual for officers. The competency standard has five components. They include elements, units, variables, criteria of performance and the underpinning knowledge. All the components should be understood and be used during assessment (Alan & Robert 2010). The CASA has a unit of competency representing a discrete function written as measures of outcomes with each unit having its own description. The units are subdivided further into elements. The elements show functions to be carried out in order to achieve a unit description, for instance by analysis of what is to be done for an airplane to safely land under all conditions, and to help in dealing with mishandled landing. Three elements emerge: landing the airplane in a crosswind, land airplane, and performing a mishandled procedure of landing. All elements have performance criteria. Performance criterion is an evaluative statement that specifies the expected performance levels, as well as what is to be assessed. Performance criteria are the flight crux standards and contain behavioral markets used as a measure of the trainees’ performance. Performance criteria always start with the verb denoting actions to be performed to demonstrate competency. This are action that one will be expected to observe from the trainee. Performance criteria that are applicable to land airplane elements include: select aiming points and identifies, selected power for idling before touchdown, flare the airplane prior to touching down at a specific height, control balloons at flare and bouncing immediately after touching down by adjusting the attitude without power application, touch down within 120/400ft meters for PPL or 60/200ft for CPL above the nominated touchdown points, Touchdown plus or minus 2 meters from the centerline, Touch down on main wheel while nose-wheel get lowered onto the runways with no harshness, maintaining directional control on the centerline, apply brakes to stop the aircraft in the distance for landing, and maintaining separation from other aircrafts. All the performance criteria should be met for one to be assessed as being competent enough to safely land an airplane as a commercial or private pilot (Alan & Robert, 2010). Range of Variables provides more definitions to the performance criteria by the elaboration of significant aspects or critical aspects of the competency unit and detailed context and conditions to be used during assessment, for instance, land Airplane variable range, must have two items. These are Day VFR and multi-aircraft or single aircraft. This means that competency can be performed in VFR during the hours of the day and multi-engine or single aircraft at day time. The assessment fails to be valid once these conditions are not met (Jerrold, 1977). However, the characteristics of CASA standards are singularly unique. This is because the Generic Range of variable applies to all standards. The Generic variables are not repeated in all the units of competence. Assessors have to apply all the conditions stated to every unit of competence they assess. Such issues are clearly clarified by the generic variables as consistency performance, temporal divergence from tolerance of the flight, the effects of turbulence on tolerance of the flight, and determining the different standards for commercial and private pilots. It is essential for assessors to use the right ROVS for all the assessments. The underpinning knowledge is applicable to competency unit not covered by CASA rating examination. This knowledge must be provided during flight training by the instructor. Candidates should interpret be able to interpret a windsock during a flight test or assessment test. For skill, behavior and knowledge, the description by competency standards and national standard are different. Instead, the words skills knowledge and attitude are used. Attitude replaces behavior because the main areas that are emphasized by CASA are assessment and training of human factors. Assessing of human factors is achieved by behavioral observations. Therefore, behavior is vital in such a case. Attitudes form a person’s mental make- up which can be speculated upon. Attitudes such as impulsiveness, anger, recklessness are always hidden or suppressed and might not be determined until behaviors such as conduct of unsafe unauthorized impulsive action, failure to follow rules are observed and exhibited. It is, therefore, necessary for assessors to look for measurable for confirmation of what could initially be a certain feeling for a person with an attitude problem (Daniel, 2010).. Assessors and instructors should be familiar with tools of the trade. Respect for this assessment tools reduces the instrumental errors. Flight standards are valuable resources for a flight assessor or flight instructor. They give details of the functions to be done, define what to be assessed, and in performance criteria, they are the behavioral makers, specifying what to assess under what condition, and to what level, as well as specifying the underpinning knowledge to be examined (Rothwell & Kazanas, 2008). Assessors have to be meticulous while applying the standards. They should pay particular attention and ensure that all the components are well considered. As an example, there are scenarios where pilots control an airplane while maintaining level flight at the level and straight flight. It also follows a process of assessing element that maintains level and straight flight. Achievement records The records were developed for certain reasons. First, it was to ensure the Day VFR syllabus for flight standards is assessed, taught, and recoded. Secondly, it was to give the trainees the opportunity that can help them review the training and be able to understand the training they are supposed to undergo (Daniel, 2010). Unfortunately, people look at achievement records as bureaucratic imposition serving no purpose but 63 individual’s signatures over a 200-hour course of flying in not demanding. However, they do not only serve as checklists of what is to be learned to acquire a license, but also serve to protect organizations of the training. Both the trainee and the instructor acknowledge through signing of documents as prove that the trainees are fully trained. It is also of importance for all the elements in the achievement records to be signed correctly. All the criteria should be competently done on more than two flights. An assessment is not an issue, especially for the practical activities such as hovering, landing, and descending, climbing, and take-off opportunities. For activities such as land and take-off, a pinnacle helicopter that may be assessed and trained by instructors on any flight, may have a different situation (Michael & Martha,1993) The next flight might not take off until a pre license test by a qualified grade one instructor signs the achievement records. In making sure, validity is upheld; records of the training must support the records of achievement. Flight records of training. They are part of the environment of aviation training. Flight records record the progresses of trainees and an area of weakness that need further developments. The records, highlights problematic areas and pant a wrong picture. However, for the CBT systems, training records does not have to document the achievements of competency during the training session. When a trainee demonstrates competency consistently, results must be recorded in the records of the training. These comments must support the completion for the achievement record. Last but not least, training records have to be comprehensive, with relevant information so that for any instructor, who needs to step in and train students, does so without doubt (Harold & Dee, 2000). Pilots’ examination and the log books are the two evidences for achieving and training. The log books have details of flying experience, types of flight training undertaken and awards of qualifications, the results of the examination record CASA ratings and licenses examination passed. Instructions for candidates and assessors and CASA test flight forms. These forms are useful resources. When used well, considerably assists the process of assessment. The items on flight report forms for the flight phases and ground must be tested. During the start of flight report test, the assessor’s instruction and candidates provides direction on conducting tests and conditions to be met. A bold print in flight test represents competency that has all the tested elements. However, a normal print denotes an element of the unit that need only be tested (Smitha & Mohan, 2008). Formative, diagnostic and summative assessment For the explanation of summative assessment and formative, diagnostic that follows, much emphasis will be on application of the methods of assessment systems and CASA CBT (Harold & Dee, 2000). Due to administrative practices that were brought in by CASA, there might be blurring of three forms of assessment. For instance, completing achievement records can be a summative assessment when it is conducted prior to the end of training. Formative evaluations or assessments monitors progress in learning during instruction and provide continuous feedback to the instructor and the trainees about what concerns success and the inability to achieve the standards. This is a type of assessment in which an instructor makes about a trainee on almost every flight. The core purpose is to help gauge progresses of the learning of the trainees and in determining whether to repeat the training exercise, or move on the next activity. For an instructor to do this accurately, he or she should be familiarize with final standards that have to be achieved and should be able to determine if the trainees are learning in accordance to the training plan. Instructors should note and seek consistence in performances. The successful performances of an element of any standard are not indication of competency, in the assessment system and CASA CBT; competences have to be demonstrated on more than two flights. Confirmation of competency to the final standards and specification of different flight can certify the training record achievements, and training records must then be annotated appropriately (Alan & Carl, 2011). A Diagnostic assessment This is the kind of assessment that measures the current skills of the trainee, the trainee’s behavior, as well as the knowledge gained at the end of learning. This is meant to help the instructor to come up with a suitable learning program for the future. It requires that an instructor understands the difficulties of the trainee in order to develop a teaching technique that ensures the trainee achieves competency. It is an evaluation conducted at the end of the whole course to ascertain if competency standards were achieved at the end of the training. This process must be meticulous, rigorous and be in conformity with the evidence rules mentioned above. Example of summative assessment and assessment systems applied to CASA CBT include the recreational pilots licenses and the GFPT (general flying tests of progress), commercial or private pilot licenses and the command instruments rating flight test (Smitha & Mohan, 2008). . Fight tests are the kinds of holistic evaluation seeking to embrace numerous elements and units. Additionally, the test seeks to confirm knowledge, skills, understanding, behaviors, as well as problem solving skill as an assessment process (Alan & Car, 2011). The test must be inter-disciplinal problem based and be a reflection of the flying activities of real life requiring theory, analysis and practice. This tests actual purpose, therefore, is to confirm that the trainee can apply all knowledge, skills and behaviors that are certified on achievement records in the practical work scenarios like the simulated charter (Tony, 2007). In conclusion, the ISD model is a fundamental tool as far as technology is concerned. It is the cornerstone to evaluation and assessment that are used in aviation training. The other components of the model are also of significance importance as so far as their application in real life situation is concerned (Michael & Martha, 1993). However, with increasing dynamism in technology, some of the components become obsolete and, therefore, there is the need for other models to be developed to address challenges that come along with change in technology. References Alan, J. & Carl, D. (2011). Implementing Safety Management Systems in Aviation, London: Ashgate Publishing, Ltd. Alan, M. & Robert, R. (2010). Managing Performance through Training and Development, London: Cengage Learning. Daniel, E. (2010). Human factors in aviation. New York: Academic Press.Rothwell, W. & Kazanas, H. (2008). Mastering the instructional design process. San Francisco: Pfeiffer. Gary, R. & Steven, M. (2010). Designing Effective Instruction, Norfolk: John Wiley & sons. Harold, O. & Dee, A. (2000). Aircrew Training and Assessment, New York: Lawrence Erlbaum Associates. Jerrold, E. (1977). Instructional design: a plan for unit and course development, New York: Fearon Publishers. Michael, S. Martha, C. (1993). Automating Instructional Design: Concepts and Issues, New York: Educational Technology. Smitha, E. & Mohan, R. (2008). Developing, Implementing and Evaluating Computer-based Instruction., Indiana: ProQuest. Tony, T. (2007). Thinking Outside ISD: A Management Model for Instructional Design, Mississippi: ProQuest. Read More