Analysis of Current Procedures in The School of Technology - Case Study Example

Introduction The School of Technology is one of eight academic schools at Oxford Brookes The School teaches, and conducts research and technology partnership in Communications and Network Engineering, Computer Science, Mathematics and Statistics, Mechanical Engineering, Media Technology, and Motorsport Engineering (Tech Brookes, 2007). The School currently implements a tedious procedure for the preparation of question papers for all its teaching departments which involve communication between the Examination and Conferment Unit (ECU), module leaders, teaching staff, internal checkers, the exam secretary, and external examiners. Due to the increasing requests of people involved in the preparation of question papers, the School is considering the computerization of the process to improve the current system by addressing the issues associated with it, while meeting quality assurance requirements in the preparation of question papers. Background Current Procedures The Examination and Conferment Unit (ECU) is a department at the Academic Registry of Oxford Brookes University which provides services to the entire university related to examinations, award certificates and conferment of awards, and award ceremonies (Brookes, 2007). The ECU has prepared Guidelines for Examination Paper Preparation and Submission (Appendix 1). At the beginning of each semester, the ECU sends the examination paper template (Appendix 2) to all module leaders in the different academic schools of Brookes, who have an examination at the end of the semester. The template is a Word document which has been laid out to comply with the University's approved format for examination papers. Upon completion of examination paper preparation by the different schools, a camera-ready hard copy of the papers along with additional exam materials such as case studies and formula sheets, should be submitted to the ECU, either by hand or using the secure pouch system. Fiona Parker is the Academic Programmes Administrator at the School of Technology. She is also the designated exam secretary and holder of the secure pouch system at the School. She prepares the TE01 and TE02 forms (Appendix 3,4,5,6) for modules that have examination and modules that only have coursework as assessment, respectively, for distribution to the module leaders at the beginning of the semester and collected together with the papers or coursework schedules. The forms are word documents which work with an excel spreadsheet in a mail merge operation (Appendix 7) which Fiona Parker needs to complete using a tedious manual process (Appendix 8). Generally, a module is taught by more than one member of the teaching staff and because of this, a question paper for a particular module is prepared by more than one member. It is the responsibility of the module leader to put together all the questions prepared by the individual members of the teaching staff. This process is tedious for the module leader as it involves collecting copies of the questions from individual staff members who currently use different editing tools or different formatting. The module leader has to re-work the formatting of the question papers so that it meets the requirements set by the ECU. When the module leaders have completed the initial preparation of question papers and solutions for each module they are responsible for, they complete the sections they are responsible for in the TEO1 and TEO2 forms. The document sets are then passed to the internal checkers in the School of Technology. The internal checkers check the papers, complete the sections they are responsible for in the TEO1 and TEO2 forms and send their comments and suggestions back to the module leaders. The module leaders then disseminate the commented documents to the individual members of the teaching staff who prepared the initial documents for them to make necessary the necessary amendments. The amended documents are then returned to the module leaders who will need to again go through the process of putting together the question paper and possibly again re-working the formatting. After the module leaders have completed the work at this stage, they pass the revised question papers and solutions along with the TEO1 and TEO2 forms to the exam secretary. The exam secretary, Fiona Parker, then reviews and checks the papers and completes the sections in the TEO1 and TEO2 forms that she is responsible for. When these are done, she sends the document sets to the external examiners who are normally senior academics charged with ensuring that the Brooke's academic standards are comparable with the standards of other UK universities. The external examiners check the question papers and solutions, complete the sections they are responsible for in the TEO1 and TEO2 forms and send their comments to the exam secretary. Upon the exam secretary's receipt of the comments from the externals, Fiona Parker again completes the forms to record comments or no comments and either get the papers amended by the module leaders or copy the exam papers and send off by secure pouch to the exams office. In case the papers receive revision comments from the external examiners, the module leaders go through the process of disseminating comments to the teaching staff a second time, and then retrieves and reformats question papers a third time to comply with ECU guidelines. At this point, the newly revised question papers and solutions are passed back to the exam secretary who will then, finally, pass the final version of the completed questions papers and solutions to the ECU, who is responsible for the management of the final copies of the questions papers. Issues and Problems A number of issues need to be addressed with the planned computerization of the System for the Preparation of Question Papers and Solutions for the School of Technology. These are: 1. The tedium of dissemination, retrieval, re-working and collation of questions and solutions prepared by individual teaching staff which is the responsibility of the module leaders. 2. The time-consuming tasks related to the preparation, dissemination and update of the TEO1 and TEO2 forms which are accomplished by the exam secretary, the module leaders, the internal checkers and the external examiners. 3. The time-consuming and therefore lack of efficiency in the means of communication between the ECU, the School staff and the external examiners. 4. The threats to the security of the exam papers due to submission by hand or the pouch system. Overall Objectives of the New System As defined by the School of Technology, the new System for the Preparation of Question Papers and Solutions should have the following objectives: 1. Address the procedural issues identified above. 2. Simplify the current procedure. 3. Enable efficient communication between the school staff, the external examiners and the university exam unit who are involved in the preparation of the question papers 4. Provide reliability, security and also ensure quality procedure of the question papers The project scope covers all procedures undertaken by the School of Technology relating to the preparation of question papers and solutions for the School, including communication with the ECU and external examiners but excluding procedures implemented by the ECU and external examiners not involving the School of Technology. Functional Requirements The functional requirements of the new system are: 1. In the preparation of questions and solutions: 1.1 For an individual member of the teaching staff to be able to add a question to the designated question paper for a particular module according to the format prescribed by the ECU. 1.2 For an individual member of the teaching staff to be able to input or attach additional materials such as case studies, formula sheets, solutions and marking schemes to the particular questions formulated in 1.1 1.3 For an individual member of the teaching staff to be able to edit a question or additional material (add, change, delete) based on revision instructions or comments from internal checkers and external examiners. 1.4 For the module leader to have browsing functionality for the questions and solutions inputted by the members of the teaching staff. 1.5 For the internal checkers to be able to input revision instructions and comments on the questions. 1.6 For the external examiners to be able to input revision instructions and comments on the questions. 1.7 For the module leader to be able to generate question paper and solution listings for the module and the related TEO form. 1.8 For the exam secretary to be able to generate question paper and solution listings for the module and the related additional materials and TEO forms. 2. In the preparation and update of TEO forms: 2.1 For the exam secretary to be able to prepare a TEO1 or TEO2 form as appropriate. 2.2 For the exam secretary to be able to enter updates to the TEO forms. 2.3 For the exam secretary to be able to send an information or reminder message to module leaders and external examiners related to the TEO forms, question papers and solutions. 2.4 For the module leaders to be able to enter updates to the TEO forms. 2.5 For the internal checkers to be able to enter updates to the TEO forms. 2.6 For the external examiners to be able to enter updates to the TEO forms. Non-functional Requirements The non-functional requirements of the new system are: 1. Accessibility - The new system should be accessible to its primary users: the teachin staff, module leaders, internal checkers, external examiners and the exam secretary. 2. Availability and reliability - The new system should be available for use by its users during the period and schedules that the users are expected to prepare the examination materials. It should be available as much as possible on 24/7 mode. The new system should also be reasonably reliable and able to perform and maintain its functions during routine circumstances as well as unexpected circumstances which may be forecasted such as a power failure. 3. Administrative documentation - The new system should have a built in system of providing information regarding information movement such as date of access and/or updates, source of access and/or updates both through the TEO forms and in a system log. 4. Maintainability - The new system should be easily modifiable in order to correct defects, meet new requirements, make future maintenance easier, or cope with a non-radical change in environment. 5. Platform compatibility - The new system should operate from a platform that is commonly used by its users. 6. Security - The new system should have built in security measures to prevent unauthorized persons from having access to information that is safeguarded, such as user IDs and passwords, user access security levels and the like. 7. Serviceability - The technical support personnel for the new system should be capable of installing, configuring, debugging and providing hardware or software maintenance to solve problems encountered by users. 8. Usability - Users of the new system should be able to easily, with minimal training and orientation, use the system, comparable to the old manual system, if not easier. Use Cases and Operation Scenarios User Profiles User Role Goal/s (Use Case/s) Exam Secretary Initiation of TEO form TEO Update 1 - Prepare TEO form Send TEO form to module leader Transmittal of question/materials to external examiners Retrieve module exams from module leaders, check submissions TEO Update (4) - Update TEO form Send TEO (4) and exam questions/materials (3) to external examiner. Transmittal of question/materials reviewed by external examiners for final revision by module leaders (option 1, if revision necessary) Retrieve module exams from external examiners, check submissions TEO Update (5) - Update TEO form Send TEO (5) and exam questions/materials (4) to module leader. Transmittal of final version to ECU Retrieve module exams, check submissions. If no comment from external examiner or completed final revision from module leader, make copies for ECU. TEO Update (6) - Update TEO form. Send TEO (6), exam questions/materials (3 or 4) to ECU. Module Leader Initial preparation of question paper and solutions Exam Template Update 1 - Prepare new exam template, input module number, header, module title, exam rubric, module leader name, footer Retrieve questions and materials submitted by teaching staff, check submissions, consolidate TEO Update 2 - Update TEO form Send TEO (2) and exam questions/materials (2) to internal checker Revision of questions/materials from internal checker instructions Review instructions/comments of internal checker. Send exam template to respective teaching staff for revision. Retrieve revised questions and materials submitted by teaching staff, check submissions, consolidate TEO Update 3 - Update TEO form Send TEO (3) and exam questions/materials (3) to exam secretary. Final revision of question/materials from external examiner instructions Review instructions/comments of external examiner. Send exam template to respective teaching staff for revision. Retrieve revised questions and materials submitted by teaching staff, check submissions, consolidate TEO Update 5 - Update TEO form Send TEO (5) and exam questions/materials (4) to exam secretary. Teaching Staff Member Add question and additional materials Exam Template Update 2 - Input exam question, attach additional materials Revision of questions/materials from internal checker instructions Review instructions/comments of internal checker. Exam Template Update 3 - Revise exam question, additional materials based on internal checker instructions/comments. Final revision of question/materials from external examiner instructions Review instructions/comments of external examiner. Exam Template Update 4 - Revise exam question, additional materials based on external examiner instructions/comments. Internal Checker Checking of module exam Exam Template Update 3 - Check module exam submitted by module leader, indicate instructions for revisions and comments in appropriate pages. TEO Update 3 - Update TEO form. Send TEO (3) and exam (3) to module leader External Examiner Checking of module exam Exam Template Update 4 - Check module exam submitted by exam secretary, indicate instructions for revisions and comments in appropriate pages. TEO Update 4 - Update TEO form. Send TEO (4) and exam (4) to exam secretary. Use Cases Use Case Update TEO1 and TEO2 forms. Description The TEO1 and TEO2 forms are for tracking the status of question paper preparation. TEO1 is used for modules with examinations and TEO2 for modules with 100% coursework only requirements for assessment. Actors Exam Secretary (ES) Module Leader (ML) Internal Checker (IC) External Examiner (EE) Examination & Conferment Unit (ECU) Assumptions The TEO forms are electronic forms which are part of the new system. Steps ES prepares new TEO1 and TEO2 forms for each academic module, and sends to ML. ML updates the forms prior to submission of questions/materials to IC. IC updates the forms prior to return of instructions/comments on questions/materials to ML. ML updates the forms prior to submission of revised questions/materials to ES. ES updates the forms prior to transmittal of questions/materials to EE. EE updates the forms prior to return of instructions/comments on question/materials to ES. ES updates the forms prior to transmittal of instructions/comments on questions/materials from EE, to ML. ML updates the forms prior to submission of final questions/materials to ES. ES updates the forms prior to transmittal of final questions/materials to ECU. Use Case Prepare details of questions and additional materials. Description Questions and Additional Materials may be prepared by several members of the teaching staff and the Module Leader. Actors Teaching Staff (TS) Module Leader (ML) Assumptions There is a standard and uniform exam template which is accessible to all designated teaching staff who will use the template for the preparation of exam questions and additional materials. Steps ML prepares the exam template for the module and provides access to designated TS. TS enters questions assigned to him/her and attaches additional materials related to the questions, such as case studies, solutions, formula sheets and marking schemes, and sends to ML. ML checks, consolidates individual submissions of TS and sends to internal checker. ML checks instructions/comments of internal checker and farms out questions for revision to TS. TS implements revisions from internal checker and sends back to ML. ML checks, consolidates individual submissions of TS and sends to exam secretary, who in turn sends to and receives back from external examiner. If there are instructions from external examiner, the exam secretary sends back questions/materials to ML. ML checks instructions/comments of external examiner and farms out questions for revision to TS. TS implements revisions from external examiner and sends back to ML. ML checks, consolidates individual submissions of TS and sends final version to exam secretary. Use Case Prepare complete modular exam/coursework materials. Description The different stages of development of the modular exam/coursework materials are the responsibility of module leaders in coordination with the checkers and administrative staff Actors Module Leader (ML) Internal Checker (IC) Exam Secretary (ES) External Examiner (EE) Examination & Conferment Unit (ECU) Assumptions There is a standard and uniform exam template which is accessible to the module leader, the internal checker, the external examiner and the exam secretary. Steps ML checks, consolidates questions and additional materials for the module and sends to IC. IC checks questions/materials and indicates revision instructions and comments, then sends back to ML. ML checks, consolidates revised questions/materials to comply with IC instructions and sends to ES. ES double-checks implementation of IC instructions then sends to EE. EE checks questions/materials and indicates revision instructions and comments, then sends back to ES. If there are no comments, ES prepares documents for submission to ECU. If there are comments, ES sends back to ML. ML checks, consolidates revised questions/materials from EE instructions then sends final version to ES. ES makes copies of final version and sends to ECU. References Malan, R. & Bredemeyer, D. 2001. Functional Requirements and Use Cases. Architecture Resources for Enterprise Advantage. Retrieved 20 October 2008, from http://www.bredemeyer.com Oxford Brookes University. 2007. Examination & Conferment Unit. Retrieved 20 October, 2008, from http://www.brookes.ac.uk/asa/exams/st_services_students.html Oxford Brookes University. 2007. School of Technology. Retrieved 20 October, 2008, from http://tech.brookes.ac.uk/ Appendix Appendix 1: Guidelines for Examination Paper Preparation and Submission Appendix 2: Examination Paper Template Appendix 3: Branded TEO1 Form Appendix 4: Branded TEO1 Form Checklist Appendix 5: Branded TEO1 Form Appendix 6: Branded TEO1 Form Checklist Appendix 7: Semester 1 Exam Papers and Advanced Courseworks for the External Examiners Appendix 8: Extraction of Transcription of An Interview with Fiona Parker Read More