Instructional Design and Program Evaluation - Essay Example

Instructional Design and Program Evaluation 1 Introduction Online/distance learning programs are rapidly gaining international popularity because they provide adult learners with the opportunity to further their education, whether for career or personal purposes, irrespective of time and space. Within the context of an increasingly and intensively competitive labor market, continued education is an essential strategy for the acquisition of a competitive edge but the primary obstacle to the acquisition of this particular competitive advantage is time constraints and geographic distance. Online education has emerged as the solution to the stated and, as such, is gaining ever-increasing popularity. Paradoxically, however, even as more and more people are expressing interest in enrollment in online programs, online learners are experiencing dissatisfaction and disappointment. There are several reasons for the stated, not least of which are the lack on interpersonal interaction and the novelty of the experience. It is precisely because of this that it is imperative that universities which deliver online courses and educational programs design and implement a best-practices framework. Doing so requires that university administrators and faculty review and evaluate their online courses for the purposes of determining extent of adherence to best practices, following from which, if needed, they should redesign the courses in question with the said best practices framework in mind. This report, focusing on MDDE 604: Instructional Design and Program Evaluation in Distance Education, will review perspectives on online learning for the purpose of forwarding suggested revisions to the course in question. 2 Course Background MDDE 604 is an online master’s course offered by Athabasca University. Students come from diverse educational, national, and linguistic backgrounds. Therefore, while the course serves Canadian students, it cannot be directed exclusively towards them since a significant percentage of students are international. The course textbooks and reading are current and relevant and the course website is easy to navigate. Course content is appropriate. The implication here is that readings, content and website do not need to be changed, or revised. The above stated does not mean to imply that the course does not withstand revisions, as it does. Revisions, however, are limited to delivery method and teaching strategies. The below, reviewing empirical and theoretical literature on online learning, contains a set of recommendations designed to enhance the learning experience and facilitate student’s understanding. 3 Online Learning A review of the literature in online learning revealed critical strategies for effective teaching practices (Palloff & Pratt, 1999 & 2000; Graham, C., Kursat. C., Byung-Ro, L., et al. 2001; Lewis, L., Farris, E., Snow, K., & Levine, D. 1999; Candiotti & Clarke, 1998; Schrum & Berge, 1998; Ladon, E. H., 2002). According to the literature, an effective instructor should focus on many factors in order to create a successful online learning environment. The following are instructor characteristics that are synthesized from the aforementioned scholars: accept the value of facilitated learning as equal to that of the traditional model; demonstrate a broad base of life experiences in addition to academic credentials; possess characteristics of openness, concern, flexibility, and sincerity; feel comfortable communicating in writing; want to introduce critical thinking into the learning process; possess the appropriate credentials to teach the subject matter; be experienced and well trained in online learning; demonstrate a very rich, active, respectful and responsive style of communication; provide a safe climate; invite input regarding the goals and agenda of the course; establish a sense of community among the learners; keep students aware of where they stand with respect to the course evaluation process; assess student learning without using tests requiring memorization; be online nearly every day; provide two types of feedback: information feedback and acknowledgment feedback; set deadlines; promote collaborative learning; encourage students to become independent learners; create places in an online environment where students can ask for help; use project-based activities instead of lectures. The above stated may be described as a best practices framework for online courses and MDDE 604 does not satisfy this framework. While the course is project based, as recommended within the context of the said framework, there is no sense of community. Communication between students is practically non-existent and minimal with the course instructor. The reason for the former is that there is no course forum. As for the second, the reason probably is that students do not feel secure or comfortable enough to engage with the instructor. This can be resolved by making the course more interactive through the addition of a curse blog or forum, where students and teachers can interact with one another. 3.1 Technological Aptitude Faculty interaction with students should be on a one-to-one and regular basis, especially for students who fall behind. If needed, the instructor should give support over the phone (Schrum, 1998). Most online faculty are online as much as seven days a week, with five being the norm. In order to avoid overload through too much information sharing, faculty should “establish policies describing the types of communication that should take place over different channels” (Graham, et al., 2001, para. 2). Examples of creating these boundaries included: “Do not send technical support questions to the instructor; send them to techsupport@university.edu.” Or: “The public discussion forum is to be used for all communications except grade-related questions” (para. 3). It is important in a distance-learning environment to establish a sense of community among the learners that will allow them to discuss, share, question, explore, reflect, and assist one another in obtaining learning outcomes (Palloff & Pratt, 1999). Schrum suggested encouraging students to post a short autobiography at the beginning of the course to make students feel they know one another. Participation guidelines help learners understand expectations. Faculty who establish minimum levels of participation in a discussion, and thus promote ongoing contributions to reciprocal knowledge building, typically have better learner outcomes (Schrum, 1998) since collaborative learning takes place. In direct reference to MDDE 604, delivery method is stilted in the sense that it is exceedingly impersonal. While the instructor answers student questions, there is no one on one relationship with the consequence being that there is no personal engagement in the course and with the material. This can be resolved, as earlier stated, through the design and implementation of a course blog/forum and active instructor encouragement of student participation, collaboration and cooperative peer-learning. 3.2 Personal/Professional Aptitude Online learning is not well suited to traditional lecture delivery. In order to enhance learning, the instructor needs to be able to accept the value of facilitated learning as equal to the traditional model. One way in which to demonstrate acceptance might be facilitated through project-oriented assignments, whereby students present case study solutions via the class Web site. The other students critique the solution and make further comments about the case. After all students have responded, the case presenter updates and reposts his or her solution, including new insights or conclusions gained from classmates. Only at the end of all presentations should the instructor provide an overall reaction to the cases and specifically comment about issues the class identified, or failed to identify. In this way, students learn from one another as well as from the instructor (Graham, et al., 2001). Regarding the goals and agenda of the course, faculty find that the more they invite learners into setting course goals and objectives, to the maximum extent possible, the more successful the learning outcomes. While they are mindful of the fact that they are content experts with clear ideas as to what students need, they also realize that learners often benefit from content that matches their individual academic and personal goals. These instructors also comfortably allow their students some measure of independence and control over their learning (Ladon, 2002). Instructors provide considerable individualized feedback through email, and threaded discussions, as well as by using various other course communication tools. Knowing that learners will be gratified by and work harder for a teacher who cares about them, they give their students a great deal of positive messages about what they are doing well. They also offer constructive criticism, provide models of good performance, and recommend links to resources for enhancing understanding of subject matter, and/or enable students to pursue material related to individual interests (Ladon, 2002). The MDDE 604 instructor can be easily contacted through the course website and, when contacted, responds to student mails. Added to that, individualized feedback is sent to all students along with their grades. Within the context of the stated, therefore, the course instructor’s professional aptitude subscribes to a best practices framework. Even though there is room for the recommendation of greater levels of individual instructor-student interaction, such a recommendation would be unrealistic and impossible to implement given class size. Therefore, instead of the aforementioned, it is recommended that the instructor promote student interaction so that learners may learn from one another. This may be done by adding collaborative and group assignments to the course, instead of just individual assignments. Group work will create a sense of community and provide learners with the level of personal interaction that they are missing from this course. 3.3 Learner Needs in an Online Environment With the massive explosion of online learning, it is critical to link how adults (most college students fall into this category) learn with how curriculum is being adapted in online learning. How online learning works, and how to teach using it, has been the focus of the majority of the research in online distance education. The learners have received much less research attention. With its roots in educational organizations, online learning has been studied extensively to determine its effectiveness when compared to traditional face to face learning. Distance education has great potential for providing rich environments for students. However, as history has taught us, new technology is not a cure-all; it has trade-offs. Morse (2003) summarized the benefits of asynchronous learning environments as flexible, participatory, accessible, and communicative. Challenges to using this environment include technology frustrations, coordination difficulties, delays in response time, and skill deficits. More student centered studies of online learning are needed to show the appropriate uses of technology and pedagogy to make this medium work for all learners. Cross (1981) categorized barriers to learner participation as institutional, situational or dispositional, whether the online situation was self-study, synchronous, asynchronous, or a combination of these delivery methods. Cross argued that it is as important to know what makes adults participate, as it is to know why they do not participate (1981). Researchers in distance education, including Thompson (1990), Biner (1993), Wegerif (1998), and Richardson and Swan (2003), raised the issue that understanding learners’ needs is critical for growth and success of online course offerings. Learner needs that can affect their online participation include learning styles, attitudes, and demographics such as age, previous education, gender, job, and family responsibilities. Thompson (1990) explored attitudes of learners who were negatively disposed toward distance education and found that they considered the restricted interactions with the instructor to be the major disadvantage. Biner (1993) examined student attitudes in distance education. The three areas studied were instruction/instructor aspects, technological aspects and course management/coordination aspects and demographic information, in an effort to understand student satisfaction. Another significant aspect of learning styles is that while students who prefer less structure can learn effectively in a more highly structured and directed environment, those who need more structure feel lost. These learners can become overwhelmed and may resort to surface learning, or drop out of a course altogether. Learning styles include preferences for various instructional techniques, learning environment preferences, thinking style preferences, and expression style preferences (Renzulli, 1999). There is a number of learning style models that describe different sets of learning preferences. They can be grouped into four general areas, according to the personal aspects on which they focus. The following delineates the four groups: Social interaction models: Perry’s Scale, Belenky’s Model, and Baxter Magolda (integration of Perry and Belenky); Information processing models: Kolb’s Learning Styles Inventory, Gardner’s Multiple Intelligence, and Gregoric Mind Styles; Instructional preference models: Dunn and Dunn’s multiple levels and Reichmann and Grasha’s learner types; Personality models: Myers Briggs Personality Indicator and Knowles’ Andragogy. Students who were accustomed, in their most recent learning experiences, to a teacher centered approach may look primarily to the teacher for information and support. Students may disbelieve information, or help provided by others, or think that the teacher is not doing his/her job. An expectation that the teacher is the subject expert is another belief that is strongly held in certain teaching and learning situations, and in other countries around the world. Chan (2001) found that these issues can affect the development of students’ independent learning. The idea of growing and learning from fellow students, and the entire learning community, can be a new concept for some adult learners. Perhaps of more importance is that the traditional ways of teaching are recognized as not being particularly effective either for adult learners and younger-generation students. In other words, when designing effective learning models, one needs to provide ways to reverse the traditional theory-to-practice approach, in order to better connect with sensing learners, who prefer to learn by direct experience, which could be relevant to adults in online learning, since adult learners also prefer concrete experience learning. The implication here is that online learning would better suit learners if it were more interactive, problem-based learning. In studying the relationship between the instructional design of learning materials and learning styles, it was found that not all learners successfully interact with various forms of hypermedia-based instructions to construct their own knowledge. Another concern is that the learners become very uncomfortable if the training is presented in a way that is unexpected or at some distance from their comfort zone. While recognizing effective learning means meeting the learner’s style, “students who are asked to function at a level higher than their current level are likely to be under a great deal of stress, especially if the two levels are not adjacent” (Woods, et al., 2000, p. 11). Learner stress also occurs when there is a mixture of learning styles in a learning situation. This mismatch between the learning experience and the student’s learning state could be an even more serious issue in online learning, where silence might be the only response. Proceeding from the above stated, a number of recommendations may be made regarding the improvement of MDDE 604’s delivery/teaching paradigm. As noted in the above, most adult learners are accustomed to the teacher-centric, lecture based learning paradigm. Within the context of online education, the tendency is more towards independent learning. This is problematic for those who are not accustomed to independent learning. Accordingly, it is advisable that the course instructor add an introductory session on learning strategies for the purpose of directing students on how to direct their own studies. Added to that, greater interaction is strongly recommended, not only for the facilitation of learning but for the satisfaction of learners’ needs to interact with others who are sharing their educational experience, for the purpose of mutual help and encouragement. As such, there should be periodic meetings between students and students and instructor, set at regular intervals. The place of meeting can be the proposed course forum. 3.4 Learner Satisfaction “Research attempting to measure the relationship of particular demographic characteristics to student success—as measured by levels of persistence and/or achievement— has resulted in often contradictory conclusions” (Thompson, 1998, p. 4). Some studies have concluded that no correlation between these outcomes and specific demographic variables such as gender (Dille and Mezack, 1991; Fjortoft, 1996), ethnic background (Dille and Mezack, 1991), or age (Powell, Conway, and Ross, 1990; Gibson and Graff, 1992). Thompson further states that “other studies suggest that certain demographic variables, perhaps not in and of themselves but rather as the markers of an accompanying set of generalized characteristics, are related to student success and/or satisfaction” (1998, p. 4). Student satisfaction was reported as a more significant predictor of dropout. Satisfaction is a subjective measure, usually self-reported by students of their impressions and feelings about the outcome (solution satisfaction) and process (process satisfaction) used to reach the course outcome (Ocker and Yaverbaum, 1999). This research of a collaborative learning situation showed that while online learners were as satisfied as face-to-face learners in the outcome/the solution, they were less satisfied with the process, which is supported in online learning research. Benbaunan-Fich, et. al (2001) found that students working in online groups were more satisfied with their solutions than individuals working on their own. Suggested benefits of group work also included reducing anxiety and uncertainty and the increased perception of learning, while the biggest disadvantage was in the quality of discussion. Satisfaction is not necessarily directly related to student performance in a course. How does satisfaction influence learners’ participation in online courses? Or should the question be “how does learner participation influence satisfaction?” Shih and Cifuentes (2001) found that the number of messages sent between online pairs was a strong indicator of the effectiveness of that group. Very effective groups had the most messages, used more instructions strategies, and used them more often. Less successful groups had fewer messages, but used a range of instructional tools. Ineffective groups showed the lowest levels of activity, and did not meet the minimum communication requirements. Proceeding from the above stated, it is strongly recommended that the course instructor create online groups. In other words, the larger class should be subdivided into groups and each group should be given a project or two to work on in collaboration. The instructor should not, however, leave the group to work in isolation but in the initial stages, should contribute to group communication so as to direct group members towards a common ground and understanding. This will give learners the same level of satisfaction they derive from the traditional learning environment – satisfaction due to the formation of social relationships with peers and a working relationship with the course instructor. This particular recommendation needs to be taken quite seriously as it directly speaks to learner satisfaction and the avoidance of the dissatisfaction which leads to non completion. 4 Conclusion Currently, many public, private, and for-profit institutions are rushing toward the development and promotion of electronic educational environments. Their motivations are quite varied, but overall student success must be an outcome worthy of attention. Many students, initially enthusiastic about online learning have found the experience disappointing. In many ways this is due to the impersonal nature of delivery and the students’ lack of familiarity with the environment in question. Course administrators can offset the stated through the incorporation of active, collaborative, and constructivist learning strategies for their online classes. This is precisely what recommended in relation to MDDE 604. 5 Bibliography Baym, N. K. (1995). The Performance of humor in computer-mediated communication. Journal of Computer-Mediated Communication, 1. Retrieved August 27, 2007, from http://www.ascusc.org/jcmc/vol1/issue2/baym.html Benbaunan-Fich, R., Lozada, H.R., Pirog, S., Priluck, R. & Wisenblit, J. (2001). Integrating information technology into the marketing curriculum: A pragmatic paradigm. Journal of Marketing Education, 5-15. Biner, P. (1993). The development of an instrument to measure student attitudes toward televised courses. The American Journal of Distance Education, 62-73. Bordia, P. (1997). Face-to-face versus computer-mediated communication: A synthesis of the experimental literature, The Journal of Business Communication, 99-120. Candiotti, A., & Clarke, N. (1998). Combining universal access with faculty development and academic facilities, Communications of the ACM, Vol. 41, No. 1, pp. 36-41. Cross, K.P. (1981). Adults as Learners. London: Jossey-Bass Publishing. Dille, B. & Mezack, M. (1991). Identifying predictors of high risk among community college telecourse students. The American Journal of Distance Education. Vol 5, No. 1 p. 24-35. Fjortoft, N. (1996). Persistence in a distance learning program: A case in pharmaceutical education. The American Journal of Distance Education, Vol. 10 No. 3, 49-59. Gibson, C. C., & Graff, A. O. (1992). Impact of adults' preferred learning styles and perception of barriers on completions of external baccalaureate degree programs. Journal of Distance Education, VII(1), 39-51. Graham, C., Kursat, C., Byung-Ro, L., et al. (2001). Seven principles of effective teaching: A practical lens for evaluating online courses. Assessment, March/April. Ladon, E. H. (2002). High touch in a high tech world: Strategies for individualizing online learning. Retrieved August 27, 2007, from http://www.ecollege.com/educator/Resources_edvoice.html Lewis, L., Farris, E., Snow, K., & Levine, D. (1999). Distance education at postsecondary education institutions: 1997-98. Retrieved August 27, 2007, from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2000013 Miller, Greenberg, E., Vojir, C., & Whitney, F. (2000). MAPP Mountain and Plains Partnership. Retrieved August 27, 2007, from http://www.uchsc.edu/ahec/mapp/voices.pdf Morse, K. 2003. Does one size fit all? Exploring asynchronous learning in a multicultural environment. Journal of Asynchronous Learning Networks (JALN). 37-55 Ocker, R. J. & Yaverbaum, G. J. (1999). Asynchronous computer-mediated communication versus face-to-face collaboration: Results on student learning, quality and satisfaction, group decision and negotiation. SwetsWise 427-440. Olaniran, B. A. (1994). Group performance in computer-mediated and face-to-face communication media, Management Communication Quarterly, 256-281. Palloff, R. & Pratt, K. (1999). Building learning communities in cyberspace. San Francisco: Jossey-Bass. Palloff, R. M. & Pratt, K. (2001). Lessons from the cyberspace classroom: The realities of online teaching. San Francisco, CA: Jossey-Bass. Powell, R., Conway, C., & Ross, L. (1990). Effects of student predisposing characteristics on student success. Journal of Distance Education, V(I), 5–19. Renzulli, J. S. (1999). What is this thing called giftedness, and how do we develop it? A twenty-five year perspective. Journal for the Education of the Gifted. 23(1), 3-54. Richardson, J. & Swan, K. 2003. Examining social presence in online courses in relation to students’ perceived learning and satisfaction. JALN. 7(1), 68-88. Ruberg, L. F., Moore, D. M., & Taylor, C. D. (1996). Student participation, interaction, and regulation in a computer-mediated communication environment: A qualitative study, Journal of Educational Computing Research, 14, 243-268. Schrum, L. & Berge, Z. L. (1998). Creating student interaction within the educational experience: A challenge for online teachers. Canadian Journal of Educational Communication Vol. 26, No. 3, pp. 133-144 Shih, Y.C., & Cifuentes, L. (2001). One tale of why and how to teach online. TechTrends, 45(6), 8-13 & 46. Thompson, G. (1990). How can correspondence-based distance education be improved? A survey of attitudes of students who are not well disposed toward correspondence study. Journal of Distance Education, V(1), 53-65. Thompson, M. M. (1998). Distance learners in higher education. In C. Gibson (Ed.), Distance learners in higher education: Institutional responses for quality outcomes (pp. 9-24). Madison, WI: Atwood. Wergerif, R. (1998). The social dimension of asynchronous learning networks. JALN 2(1), 34-49. Woods, D., Felder, R., Rugarcia, A., & Stice J. (2000). The future of engineering education. Chemical Engineer Education, 34(2). Read More