MCQs Assessment Method in the Medical School - Report Example

Name: Tutor: Title: MCQS Assessment Method In the Medical school Course: Date: MCQS Assessment Method In the Medical school Introduction Assessment is an important component of the education learning process. It acts as the major driver and motivator of students’ learning. When an assessment focus is on recall of accurate knowledge, student will be inclined to adopt a surface approach but when the assessment calls for high level of cognitive capabilities, students will be more inclined to espouse a deep approach (Scouller 1998). The learner’s perceptions of the education environment greatly influence their learning. Therefore, the way an assessment is conducted influences the students’ choice of studying approach. There are many methods of assessment that are used in the medical school to assess the success of students. Multiple Choice Questions (MCQs) is the most widely used assessment method in the medical field. It has been used as an assessment method for more than 50 years. There are several reasons that have led to its popularity, including being east to administer and mark and have high testing efficiency and objectivity (Schuwirth and van der Vleuten 2011). This paper set out to assess for the reliability, validity, feasibility, cost and learning impacts of MCQs. What MCQs Can Assess There is a wide range of students’ knowledge that a MCQs can assess within a short period of time. Since many MCQs may be created to cover the content of a given topic, it allows for a wide coverage of concepts which may be tested consistently. Hence, the MCQ assessment method allows for high level of test reliability. MCQS can also allow for higher levels of test validity if they are taken from a representative section of content area which constitute set learning outcome (Khan and Aljarallah 2011). Opponents of MCQs do not agree that MCQS can test advanced level learning. But this criticism is frequently credited to errors in the developing of the items instead of their intrinsic weakness. If MCQs are appropriately developed, they lead to objective testing which may assess knowledge, comprehension, analysis and application. The shortcomings of MCQs are that they assess for recognition instead of recall, permit for guessing and are hard to construct and hence, time and effort-consuming (Reid, Duvall, and Evans 2007). What constitute a quality MCQs? There are several types of MCQs that are used as a form of assessment in the medical school. The most popular types include the A-Type and the R-Type. Normally, the A-Type has 4-5 choices for the learner to select from while the R-type or Extended Matching Questions entails giving a theme for every question and learners are supposed to match the choices provided, normally 3-26, with the Scenarios, normally 5-10. The A-type question is made up 3 elements: the stem, the item; the stem and the options. The item refers to the whole unit and comprise of a stem and numerous choices. The stem refers to the question or the lead-in question. Options are the possible answers, also known as choices or alternatives. Keyed response refers to the correct option whereas foils refer to the wrong option (Al Muhaidib 2010). To come up with good MCQs, the teachers need to identify the most essential information or skill which needs to be learned. There must be a direct association between instructional objectives and test items. Therefore, test items must come straight from the objectives and emphasis on essential and relevant information, evading testing the comprehension of medical finer points. Contentious items must be evaded, particularly when the information is deficient or the information is disputed (Downing 2002). Reviewing the most important subtopics of the content as well as finding out sentences which summarize major ideas can aid in developing appropriate questions. From here, the teacher needs to write down key facts as declarative sentences so as to make it clear to the students of what is expected of them (Downing 2006). It is worth testing the written idea if it forms an essential element of the instruction. Objectives must be set as per the specific student behavior and according to what the program is supposed to teach. They must define essential knowledge as well as supported by the instruction given through the learning program. This is critical as the student that be assessed whether s/he has achieved an objective if the teacher come up with measurable objectives. Perfect examples of measurable terms include explain, list, state and identify. Examples of non-measurable terns are learning, know, understand and become familiar (Gronlund 2009). Threats to validity The validity test of MCQs has many threats that can be divided into 2 headings. One is the Construct Irrelevance variance (CIV) that is the variable which methodically interfere with the capability to meaningfully interpret marks. Such variables include flawed items, statistically biased items, cheating, inappropriate reading level, teaching to test and test wisdom. The other threat is content under-representation (CU) that is the under sampling of the content. It is the role of the teacher to ensure that these threats as well as their associated factors are reduced to minimum so as to have a fair and valid test. The CIV threat can be avoided by making sure that the questions covers the essential content and also constructing good questions that are easy to understand. The major challenge to construction of good questions is that there are not many teachers in the medical school that are formally trained in questions constructions. The other challenges that question construction face are related to the items’ construction process. In most countries, medical school students are supposed to sit for exams for at least 3hours after every 4-6 weeks in their life-time in college. Ironically, a lot of questions are constructed during the last minute and are also constructed by many teachers who taught the course and therefore there is limited time available to review the questions to ensure overall quality before student sit for the exam. Moreover, there might be no conformity on the standard of constructing the questions (Jozefowicz, Koeppen and Case 2002). Recommendations Despite the importance of the blue print test in eliminating the CIV threat in questions constructions, research studies have found that only a number of medical schools are applying it for validating test due to the fact that teachers do not have the required skills. Therefore, it is suggested that teachers should be trained on how to develop questions and use blue print tests. There are many publications as well as simple guides that have been developed which can be used in such training. To enhance the validity of MCQs, the administrations in medical schools should develop a faculty programmes that emphasis on assessment validity, adopt a blue print process for questions construction and develop or adopt guidelines for construction of item (McKinley and Norcini 2013). The use of blue print test can be used to avoid the content under representation threat. The blue print test is a phrase that has been taken from the architecture field, which indicates that the assessment process should be carried out as per a replicable plan. The utilization of the blue print test will make sure that the test is constructed and mapped cautiously against course’s learning objectives, guaranteeing just representation of objectives (Hodges 2013). The impact of Item-writing Flaws on learners’ performance MCQS are not free of item-writing flaws. Research study has found 60% of the constructed MCQS in textbooks have item-writing flaws or violates at least one of the writing guidelines (Downing 2005). Downing SM. (2005).The most common item-writing flaws include making the item too easy or very difficult. Technical flaws are divided into two groups: irrelevant difficulty and test wiseness. Irrelevant difficulty flaws include vague terms, ambiguous information, poor order in numeric data, unnecessary information, and omission of lead-phrase in the stem and lack of a correct option. Test wiseness flaws include grammatical errors, logical cues, absolute terms, word repetitions, implausible distracters, convergence, longest choice is the correct, none of the above as well as All of the above. Research studies have also found out that most students who fail as a result of flaws in questions could have passed were the questions constructed appropriately (Tarrant, Knierim, and Hayes 2006). How to avoid Item-writing flaws There are several ways in which item-writing flaws can be minimized. One way is to make sure that the stem is short and contain only the important content. Teachers should not use the stem to teach or else include statements that are informative but unnecessary for selection of correct choice. Moreover, the stem must not be misleading or tricky as it may deceive the student to select the incorrect option. Teachers should also use simple language to construct stems as this is not an opportunity to test the students’ reading ability. It is usually estimated that students should answer 1-2 questions in a minute, so items that considerably go beyond this time to answer must be closely evaluated to find out if they contain unnecessary information or are confusing (Pepple, Laurine and Carroll 2010). The second way of avoiding item-writing flaws is by stating the stem in a manner that just one choice may be validated and that choice must be indisputably correct. It is important to document the source of the stem validity. The third way of reducing item-writing flaws is by constructing questions that only solicit for the correct options rather than the wrong answers. Negative questions should be used selectively as they are difficult for student to comprehend and hence are less effective. When using negative stem, it is important to make sure that the negative term is underlined, capitalized or else italicized to ensure the student sees it (Biggs 2003). For example, “Which of the following is NOT a cause of heart failure?” The other way to eliminate item-writing flaws is by avoiding using absolute terms, such as always, all, none or never in the stem. Also, vague terms, such as occasionally, sometimes, seldom, few, many and rarely should be avoided as they are not homogeneously understood. Furthermore, the use of acronyms, eponyms or abbreviations should be avoided as students might not be familiar with such phrases or the phrases may have many meanings (Schuwirth and van der Vleuten 2006). The major challenge to developing quality MCQs is designing plausible distractors. The capacity of the item to categorize the examinees into those who know and those who don’t know is based on the quality as well as attractiveness of the distractors. Statements that are correct but do not completely meet the problem’s requirements and incorrect statements which seem correct to the examinee are the best distractors. Every incorrect choice must be plausible but visibly incorrect. Implausible, nonsense or trivial distractors must not be used. Distractors must be related or else associated to each other. This means that they must belong to the same group as the correct option. Distractors must be related to the correct option in terms of length, grammar and complexity (Tarrant and Ware 2012). The perfect number of choices is 3-5. A Study has demonstrated that three-choice items are as efficient as four-choice alternatives (Case and Swanson 1998). Developing more that five options can be time and energy-consuming and frequently result in incorrect choice and at the same time increasing the examinee’s reading demands. Moreover, choices should not contain “all of the above” or “none of the above”. In addition, choices must be arranged in a logical order. For instance, if the choices are dates they must be listed in a chronological order. Choices must also be independent; they should not be related to each other (Abdel-Hameed, Al-Faris and Alorainy 2005). Grammar cues should also be avoided as much as possible when constructing questions. The examiner should make sure that there is tense as well as subject-verb agreement between the stem and the options. Also, each item should be independent so as one it does not reveal content which allows the student to automatically identify the correct answer of another item. Moreover, it is important to avoid a question that requires the examinee to know the correct answer so as to be able to answer the next question. The keyed response’s position must differ from the A, B, C and D positions as most study has shown that the position B and C has been overused. This is because test wise examinees, who are familiar with this trend, will select B or C so as to increase their possibility of getting the correct answers in case they are unable to identify the correct answer and are left to guess (Struyven, Dochy and Janssens 2005). Conclusion The most widely used assessment method in the medical school is the MCQs. This is because MCQs are relatively easy to administer and grade and have high testing efficiency and objectivity. However, MCQs have some disadvantages, including being difficult to construct and hence time and energy consuming. There are also several threats to construction of quality MCQs, which are grouped into 2 groups: Construct Irrelevance variance (CIV) and the Content Under-representation. The major challenges to construction of quality MCQs include lack of the required skills on part of the teacher as well faults of the exam constriction process itself. It is therefore critical to train teachers on how to develop well-structured questions using the text books and writing guidelines available. Teachers should also review the MCQs to make sure that they are free from item-writing flaws so as to ensure validity and reliability. Bibliography Abdel-Hameed AA, Al-Faris EA, Alorainy IA. (2005).The criteria and analysis of good multiple choice questions in a health professional setting. Saudi Med Journal, vol;26(10):1505– 1510. Al Muhaidib NS (2010). Types of Item-Writing Flaws in Multiple Choice Question Pattern- A comparative study. Journal of Education Psychology Sci.vol, 2(2):9–45. Biggs J. (2003). Teaching for Quality Learning at University. Philadelphia: The Society for Research into Higher Education & Open University Press. Case S, Swanson D. (1998) Constructing written test questions for the basic and clinical sciences (second edition).National Board of Medical Examiners, Philadelphia (available free on the web: http://www.nbme.org). Downing SM. (2005). The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Advanced Health Science Education, vol, 10(2):133–143. Downing SM. (2002). Construct-irrelevant variance and flawed test questions: do multiple- choice item-writing principles make any difference? Acad Med,77(Supll):103–104. Downing SM. (2006). Twelve steps for effective test development. In: Downing SM, Haladyna TM, editors. Handbook of test development. Mahwah NJ. Lawrence Erlbaum Associates Publishers. pp. 3–25. Gronlund NE. (2009). Assessment of student Achievement (8th ed) Boston: Allyn& Bacon. Hodges B. (2013). Assessment in the post-psychometric era: learning to love the subjective and collective. Medical Teacher 35:7 p564-8 Jozefowicz RF, Koeppen BM, Case S. (2002). The quality of in-house medical school examinations. Acad Med. Vol.77 (2):156–161 Khan MZ, Aljarallah BM. (2011). MEQs and MCQ as a tool for assessing the cognitive skills of undergraduate medical students. Introduction Journal of Health Science, vol;5(1):45–50. McKinley DW, Norcini JL (2013). How to set standards on performance-based examinations; AMEE GuideNo.85.. Medical Teacher 36:2 p97-110 Pepple DJ, Laurine YE, Carroll RG. (2010). A comparison of student performance in multiple- choice and long essay questions in the MBBS stage I physiology examination at the University of the West Indies (Mona Campus) Adv Physiol Education, vol 34(2):86–89. Reid WA, Duvall E, Evans P. (2007). Relationship between assessment results and approaches to learning and studying in Year Two medical students. Med Education, vol, 41:754–762. Schuwirth LWT & van der Vleuten CPM. (2011). General overview of the theories used in assessment: Medical Teacher 33:10, pp783-797. Schuwirth LWT & van der Vleuten CPM. (2006). How to design a useful test. ASME, Edinburgh Scouller K. (1998). The influence of assessment method on students’ learning approaches: multiple-choice question examination versus assignment essay. Higher Education. Vol. 35:453–472. Struyven K, Dochy F, Janssens S. (2005). Students’ perceptions about evaluation and assessment in higher education: a review. Assess Evaluation Higher Education, vol, 30(4):325–341. Tarrant M, Knierim A, Hayes SK. (2006). The frequency of item writing flaws in multiple- choice questions used in high stakes nursing assessments. Nurse Education Practice. 6:354–363. Tarrant M, Ware JA. (2012). Framework for improving the quality of multiple-choice Assessments. Nurse Educator. Vol 37(3):98–104. doi: 10.1097/NNE.0b013e31825041d0. Read More

To come up with good MCQs, the teachers need to identify the most essential information or skill which needs to be learned. There must be a direct association between instructional objectives and test items. Therefore, test items must come straight from the objectives and emphasis on essential and relevant information, evading testing the comprehension of medical finer points. Contentious items must be evaded, particularly when the information is deficient or the information is disputed (Downing 2002).

Reviewing the most important subtopics of the content as well as finding out sentences which summarize major ideas can aid in developing appropriate questions. From here, the teacher needs to write down key facts as declarative sentences so as to make it clear to the students of what is expected of them (Downing 2006). It is worth testing the written idea if it forms an essential element of the instruction. Objectives must be set as per the specific student behavior and according to what the program is supposed to teach.

They must define essential knowledge as well as supported by the instruction given through the learning program. This is critical as the student that be assessed whether s/he has achieved an objective if the teacher come up with measurable objectives. Perfect examples of measurable terms include explain, list, state and identify. Examples of non-measurable terns are learning, know, understand and become familiar (Gronlund 2009). Threats to validity The validity test of MCQs has many threats that can be divided into 2 headings.

One is the Construct Irrelevance variance (CIV) that is the variable which methodically interfere with the capability to meaningfully interpret marks. Such variables include flawed items, statistically biased items, cheating, inappropriate reading level, teaching to test and test wisdom. The other threat is content under-representation (CU) that is the under sampling of the content. It is the role of the teacher to ensure that these threats as well as their associated factors are reduced to minimum so as to have a fair and valid test.

The CIV threat can be avoided by making sure that the questions covers the essential content and also constructing good questions that are easy to understand. The major challenge to construction of good questions is that there are not many teachers in the medical school that are formally trained in questions constructions. The other challenges that question construction face are related to the items’ construction process. In most countries, medical school students are supposed to sit for exams for at least 3hours after every 4-6 weeks in their life-time in college.

Ironically, a lot of questions are constructed during the last minute and are also constructed by many teachers who taught the course and therefore there is limited time available to review the questions to ensure overall quality before student sit for the exam. Moreover, there might be no conformity on the standard of constructing the questions (Jozefowicz, Koeppen and Case 2002). Recommendations Despite the importance of the blue print test in eliminating the CIV threat in questions constructions, research studies have found that only a number of medical schools are applying it for validating test due to the fact that teachers do not have the required skills.

Therefore, it is suggested that teachers should be trained on how to develop questions and use blue print tests. There are many publications as well as simple guides that have been developed which can be used in such training. To enhance the validity of MCQs, the administrations in medical schools should develop a faculty programmes that emphasis on assessment validity, adopt a blue print process for questions construction and develop or adopt guidelines for construction of item (McKinley and Norcini 2013).

The use of blue print test can be used to avoid the content under representation threat. The blue print test is a phrase that has been taken from the architecture field, which indicates that the assessment process should be carried out as per a replicable plan.

Read More