The Potential of Student Portfolios in Radiologic Education - Assignment Example

The Potential of Student Portfolios in Radiologic Education Introduction A portfolio can be defined as, "a purposeful collection of student work that exhibits to the student (and/or others) the student's efforts, progress, or achievement in (a) given area(s). This collection must include student participation in selection of portfolio content, the criteria for selection, the criteria for judging merit, and evidence of student reflection.” (Reckase, 1995, p. 12) Review of Literature Types of Portfolio Documentation Portfolio or working portfolio: The process involves collecting work over a period of time. This will show the growth and improvement of a student’s work and reflect the learning process. Everything from brainstorming activities, drafts and finished products can be included in the documentation portfolio. Specific items can be selected, so as to focus on particular educational experiences or goals. It can include the best and weakest of student work. (Paul SG, 1995) Process Portfolio: This documents the students' overall learning process. It can show how students integrate specific knowledge or skills and progress towards both basic and advanced skills. The portfolio emphasizes the students' reflection of their learning process. “The portfolio can include the use of reflective journals, think logs, and related forms of metacognitive processing.”( Paul SG, 1995) Showcase Portfolio: This type of portfolio should include the students' best, completed work. “In addition, this type of portfolio is especially compatible with audio-visual artifact development, including photographs, videotapes, and electronic records of students' completed work. The showcase portfolio should also include written analysis and reflections by the student upon the decision-making process(es) used to determine which works are included.”( Paul SG, 1995) Characteristics of an effective portfolio 1. “It is continuous and ongoing, providing both formative (i.e., ongoing) and summative (i.e., culminating) opportunities for monitoring students' progress toward achieving essential outcomes. 2. It is multidimensional, i.e., reflecting a wide variety of artifacts and processes reflecting various aspects of a students' learning process(es). 3. It provides for collaborative reflection, including ways for students to reflect about their own thinking processes and metacognitive introspection as they monitor their own comprehension, reflect upon their approaches to problem-solving and decision-making, and observe their emerging understanding of subjects and skills.”( Paul SG, 1995.) The minimum components of a personal learning portfolio (Judy McKimm, 2001) 1. Title page. 2. Index. 3. Log book. 4. Map of learning experiences 5. Case reports. 6. Appendices. 7. Useful references. Depending on the purpose of the portfolio, they can also include: 1. A reflective diary/log/journal. 2. Selected pieces of work such as elective reports, video consultations. 3. Reflective commentaries on critical incidents. 4. Learning plans. 5. Clinical procedures. 6. Copies of assessment and feedback forms eg. clinical attachments, and lab work. 7. Project work. 8. Critical reviews of articles Self-Reflection and mentor relationship Self-reflection is an essential component of portfolios, and a critical skill needed in practice. However, students are given little opportunity to learn or practice self-reflection during their education. (Westberg J, 2001). Students need to be taught how to reflect. Students are generally not very motivated to complete reflective portfolios. Usually, only students given to personal reflection are motivated to complete a reflective portfolio. Most students need the external motivation of a course or program requirement to complete them. The appeal of portfolios is that they use a student's actual work and require reflection on the part of the student. Therefore, portfolios have the potential to assess more complex skills than other measures (Patricia OS, Constance G, 2002, pp. 1305-1309) Portfolios can enhance the relationship between residents and faculty members. “Faculty can work with the residents in identifying interesting cases, pointing out anomalies, and providing feedback. In mentoring relationships, portfolios may shift more toward formative feedback and less toward assessment.” ( Patricia OS, Constance G, 2002, pp. 1305-1309) Evaluating clinical competence Portfolios can evaluate competence in a clinical area. For example, a student might mention about a case seen in the Radiology Department. The student can note the history of the patient and the physical examination findings in the first document. The second document for this portfolio could be a short evaluation by the supervising faculty, attesting as to how the case was presented (organization, conciseness, completeness), the differential diagnosis and final diagnosis, the plan (workup and management), the interpretation of results, the timely response and accuracy, and disposition decision. This might enhance the teacher-learner interaction. The student could then summarize the patient's course. A self-reflection about how well the student demonstrated the six general competencies constitutes the final portfolio document. (Patricia OS, Constance G, 2002, pp. 1305-1309) A "follow-up" portfolio entry might include: 1. A copy of the patient record. 2. A description of the knowledge and/or skills deficits identified through the case. 3. The plan developed to address these deficits. 4. A self-reflective statement explaining how the resident identified the deficits, details of what was learned, and a description of how behavior will change, based on this learning effort. (Patricia OS, Constance G, 2002, pp. 1305-1309) Traditional versus Web-based portfolios. Traditional paper portfolios Paper portfolios are an established way to showcase a person’s work. However, some of the major drawbacks to paper portfolios include the requirement of physical space to store them, the difficulty in providing access to multiple readers, and delivering them to readers who are not close to the storage location. “The media itself is limiting. Even though audio and videotapes can be included, the requirements for viewing (i.e., set-up time and equipment) make them cumbersome. The process of revision based on reviewer comments also is cumbersome.” (Leah MH, Silva K) Web-based portfolio or electronic portfolio A webfolio can be defined as “a tightly integrated collection of Web-based multimedia documents that includes curricular standards, course assignments, student artifacts in response to assignments, and reviewer feedback to the student’s work.”(Paul G, Douglas L et al, 2002.) The electronic logbook Traditionally, the clinical experience of a student radiographer has been recorded in a paper logbook. “The electronic logbook is a method of accurately recording the clinical experience of the student radiographer by retrieval of information from the database of a hospital or radiology management system. The system requires minimal software modification. Retrieved information is processed by a specially designed software package at the Academic Center and reports issued to clinical tutors. Benefits of the system include easy monitoring of student workload, ensuring that the student receives a wide and varied experience during training, improvement in student motivation, and use of a final report form in the student's portfolio.” (Akehurst JC, 1999. pp. 76-79.) Advantages of a web-based portfolio (Paul G, Douglas L et al, 2002.) 1. Elimination of storage problems associated with traditional portfolios. 2. The student can house artifacts in a virtual environment. 3. Instructors can view the portfolio and comment on the artifacts. 4. The student can send an e-mail message with an enclosed URL and anybody in the world can access the webfolio. 5. The student has complete control over what artifacts are displayed and who can see them. 6. They can have a working portfolio, (can be viewed only by them), a developmental portfolio, (can be shared with the faculty), and a showcase portfolio, (can be shared with the world). 7. The student can confer and collaborate with anyone in the world. 8. Graphics, sound, digital video, text and other presentation media can be presented. Limitations of a webfolio 1. Special hardware and software is required to create these portfolios and there is no central storage place available for all the CDs. 2. Dissemination and revision are difficult with CDs. Multiple copies may be disseminated to readers, but subsequent revisions make the copies obsolete and the dissemination process must be repeated. Creating portfolios on the Web as a sample of student work alone is not enough. In order to realize the full realization of portfolio benefits, the Webfolio system must organize the entire portfolio related data in a database. (Leah MH, Silva K) The potential of student electronic portfolios to improve learning lies not just in the design, but, even more importantly, in those who will be using them. It must be modifiable and customizable in order to suit the educational values of the institution, and the faculty and students in the institution. “Student electronic portfolios, by providing authentic evidence of student learning are beginning to play a significant role in the assessment of teaching and learning.” (Sharon JH, 2003, pp. 283-303) Portfolio Evaluation The portfolio of a student's actual work helps in promoting self-directed learning and self-identification of gaps in their education, performance or level of competency. The student is able to demonstrate practice-based learning by choosing a particular case and by justifying why it is a good case. When used along with activities like patient follow-up, “this tool can give the student more flexibility and autonomy in determining their individual educational activities, enhance satisfaction with residency education, and promote the habits residents need as independent practitioners addressing continual medical education.” (Patricia OS, Constance G, 2002, pp.1305-1309) The criteria by which the faculty evaluates a portfolio might include: 1. A justifiable reason for following up the case and likelihood that the resident would learn from this follow-up. 2. Completeness of the information on the follow-up. 3. Learning plan developed and resources chosen. 4. Reflective statement indicating practice-based learning. At least one faculty member, who has knowledge of these guidelines and trained to evaluate the entry, analyzes the entry. “The final assessment is global: exceeds expectations, meets expectations, and does not meet expectations. The evaluator assesses the portfolio entry, not the original performance. Written guidelines are needed, including the scoring criteria, and faculty training would be required to score such portfolio entries. Ultimately, the program should validate that such portfolio performance relates to other measures of medical knowledge, such as examinations. Similarly, scores from this type of portfolio entry might relate to peer and student ratings of teaching and communication, further providing evidence of validity.” (Patricia OS, Constance G, 2002, 1305-1309.) Reliability of assessment criteria Although the potential of portfolios are numerous, there are however, difficulties associated with the lack of standardization of the portfolio in terms of developing assessment instruments. “As long as formal assessment is based on the philosophy of measurement, portfolios will be difficult to assess and will require the development of noncomparative methods of assessment.” (David S, Mary T, 1998. pp.192-199). There are arguments that portfolios should not be assessed summatively because there is little evidence supporting the reliability of their assessment. A study was performed to determine the reliability of assessment criteria used for a portfolio at the University of Nottingham. (Rees CE, Sheard CE, 2004.) In this study, two independent analysts assessed a random sample of portfolios using criterion-referenced assessment. “Students' performances were examined against subjective items in five areas: portfolio structure, level of critical reflection, level of skills development, use of documentary evidence, and use of relevant literature. These subjective judgments were later converted into quantitative scales ranging from 0 to 3 so that interrater reliability could be established. The level of agreement between the two analysts for the total percentage score was established using an intraclass correlation coefficient and for the individual items using weighted kappa coefficients. The findings suggest that discussion and negotiation between independent assessors can enhance the reliability of assessment criteria. Therefore, medical educators are encouraged to use such procedures in the summative assessment of portfolios.”(Rees CE, Sheard CE, 2004. pp.138-44) Criteria for Judging Merit A holistic scoring approach to evaluate a portfolio entry has been described (Patricia OS, Ed.D, Kay KA et al. 2002). “Holistic ratings require expert judgment to analyze the rather complex tasks involved in a performance such as a portfolio entry. The raters look at each entry overall rather than analyzing constituent skills or tasks, which would require assigning points to specific components. For holistic scoring, the committee developed a general scoring rubric, the specific set of rules used to score an open-ended task such as a portfolio entry. They chose a rubric focused on competency. The rubric defined resident competency in six levels: 1.Important skills and/or knowledge missing to adequately deal with the case. 2.Appropriate approach to case, but has weaknesses that could lead to potential problems. 3.Competent to successfully treat standard cases following a conventional approach. 4.Competent to successfully treat complex cases following a conventional approach. 5.Well-integrated knowledge and skills to resolve complex problems. 6. Wide breadth of knowledge and skills integrated from numerous and varied sources to resolve very complex cases.” (Patricia OS, Ed.D, Kay KA et al, 2002, pp.173-179) Student Portfolios: Administrative Uses While there is a growing understanding of instructional uses for portfolios, they are increasingly being called upon to serve administrative functions as well. Student portfolios are being used for accountability reporting, program evaluations, and a variety of administrative decisions affecting the future of individual students. (Education Consumer Guide.) Conclusion Portfolios provide educators a method to enhance, document, and evaluate the clinical learning of a student. They can enhance the assessment process by revealing the students’ skills and range of understanding. They reflect change and growth of the student’s activities over a period of time and encourage student, teacher, and parent reflection. Portfolios stimulate self-reflection, which is a critical for long-term learning and evaluation. However, there is still a lack of standardization in the assessment criteria and more research and development is needed in this area. ************************************************************************ REFERENCES Akehurst JC, 1999. Electronic monitoring of clinical experience during undergraduate training in diagnostic radiography . The British Journal of Radiology, Vol 72, Issue 853 76-79. David S, Mary T, 1998 . The use of portfolio learning in medical education. Medical Teacher, Volume 20, Number 3. pp.192 – 199. Education Consumer Guide. Office of Research, Office of Educational Research and Improvement (OERI) of the U.S. Department of Education. Judy MK, 2001, Using portfolios in medical education, Imperial College School of Medicine. Retrieved October 22, 2005. http://www.medev.ac.uk/ltsn_images/pdfs/portfolios.pdf Leah MH and Silva K, Special Education Teacher Preparation and the Electronic Portfolio. Retrieved October 24, 2005. http://jset.unlv.edu/18.1/asseds/smith.html Patricia OS, Constance G, 2002. Portfolios: Possibilities for Addressing Emergency Medicine Resident Competencies. Acad Emerg Med .Volume 9, Number 11. pp. 1305-1309 Patricia OS, Ed.D, Kay K et al , 2002. Portfolios as a Novel Approach for Residency Evaluation. Academic Psychiatry 26. pp.173-179) Paul G, Douglas L et al, 2002. On implementing Web-based portfolios. Retrieved October 24, 2005. http://www.educause.edu/ir/library/pdf/eqm0224.pdf Paul SG, 1995. What Is Portfolio Assessment Really and How Can I Use It in My Classroom? Gainesville, FL: Teacher Education Resources. Retrieved October 24, 2005. http://www.pgcps.pg.k12.md.us/~elc/portfolio1.html Rees CE, Sheard CE, 2004. The reliability of assessment criteria for undergraduate medical students' communication skills portfolios: the Nottingham experience. Med Educ. Feb;38(2). pp.138-44. Reckase, 1995. Portfolio assessment: A theoretical estimate of score reliability. Educ Measure Iss Pract.;14. p.12. Sharon JH, 2003. A Principle-Based Approach to Assessing General Education Through the Majors. The Journal of General Education - Volume 52, Number 4, pp. 283-303 Westberg J, 2001. Helping learners become reflective practitioners. Educ Health ;14. pp. 313 –21. Read More