Oral Communication Assessment in ESL Students - Research Proposal Example

Factors Affecting the Oral Communication Assessment in ESL Introduction Second language learners try their best to become accepted in the universities in both the United States and the United Kingdom in order to become globally competitive in their fields of study. However, they will not be accepted within these universities unless they have passed the English proficiency level that the university requires of them. Therefore, ESL students try their best to become proficient in the English language. Second language learners are doing their best not only to learn the English language but to become proficient with it at the same time. They enrol themselves into various English proficiency programs that would help enhance their skills in order to become competitive in the global market. However, there are several issues that are thrown against the ways in which the English language are taught. Perhaps one of the most significant issue that concerns the study of the English language among these students are the ways in which oral communication is taught as distinct from the written communication. Moreover, these two major forms of communication have significant uses in which they are utilized appropriately. The oral communication which involves the speaking ability of the person is utilized most of the time as he or she is required to speak upon personal interaction with various people. One is expected to exhibit proficiency in the oral communication such that it is one of the most common forms of interaction. The inability to showcase good speaking skills creates an impression of incompetence. On the other hand, the written form of communication is yet another major form, however, not as often used as the oral form. Mostly utilized in academic and creative aspects of communication, it is considered that the written form of communication is the ones that focus more on the technical aspect of the English language because there is enough room to evaluate the language’s technical aspects such as grammar and sentence construction. The Oral Communication and its Proficiency Test Oral communication is observed to be the most and the widely used form of communication, thus lies the need to help student in this area is considered to be greater than in the written form. In one’s daily undertakings, he communicates more orally then in the written form. Therefore, the use of the English language in oral communication must be given as much attention that is necessary especially among English as a second language (ESL) learners. Being able to communicate proficiently in using the English language had become a measure of an ESL student’s communicative competence. As the dominant form of communication in one’s daily interaction, oral communication – in the form of conversation – a person is able to demonstrate his skill and ability to participate actively and appropriately in group discussions. This communication skill is what second language learners would need to achieve, not alone for competitiveness in the job market, but for self-confidence as well. (Kormos, 1999) Competency in participating actively and appropriately in a group discussion encourages various rooms for improving ones oral communication skills. When a second language learner is able to take part in a group discussion, he is provided with an overwhelming opportunity to show his skills in a conversation. According to Kormos (1999), conversational competence is illustrated in oral communication into the various situations. If a person is able to “perform openings, re-openings, closings and pre-closings, establish and change topics, hold and yield the floor, backchannel, interrupt and to collaborate, as well as recognize and produce adjacency pair,” therefore he has mastered the art of conversation. When a second language learner is able to perform these factors that are deemed important in a conversation, he is then able to exhibit competency in oral communication. (Kormos, 1999) On the other hand, Annie Brown (2003) made a contrasting opinion regarding conversation as a method of assessing the proficiency of the speaking skills of a second language learner. A critical characteristic of conversation identified by Brown (2003) is its unstructured format for which there is no significant guideline that would suggest topics and focus of the communication. On the other hand, she also recognizes that conversation is also a valid measure in assessing the proficiency level of ESL candidates due to its spontaneity. (Brown, 2003) Nevertheless, according to Brown (2003) there is a need for the interviewers to undergo training in order to manipulate the proficiency tests accurately such that they will be able to ask relevant and significant questions, as well as provide appropriate responses that are at the same time the most basic requirements from the candidates in order to obtain focus in the conversation. The growing number of ESL students in the United States and United Kingdom has attracted the researchers, either Americans, Europeans, and even second language learners, to attend to the various proficiency tests conducted with L2 students. In order to identify the needs of the ESL students’ and their proficiency in the oral communication skill, the most common measure or method used is the interview. As the most basic form of analyzing the students’ need, interview has attracted the language researchers in finding out whether this method provides a valid result that is reliable and dependable. Measuring a person’s proficiency and competence in his participation in the oral communication is a challenging one, especially in the case of second language learners or bilinguals. (Sanchez, 2006) Formally referred to as the Oral Proficiency Interview (OPI) in proficiency tests for second language speakers such as Test of English as a Foreign Language (TOEFL), International English Language Testing System (IELTS), General English Proficiency Test (GEPT), and American Council on the Teaching of Foreign Languages/Educational Testing Service (ACTFL/ETS), OPI has attracted as many researchers in identifying the factors affecting its liberation. Among other studies conducted with regards to measuring the level of competency and the proficiency of the second language learners, Kormos’ (1999) research may be found to be one of the most significant and the most critical one. Kormos (1999), in her study regarding the use of OPI and role-plays in the language proficiency tests, stated that interviews are unequal social encounters that is not an appropriate measure in assessing the proficiency level of L2 students. The traditional method may however become appropriate in measuring linguistics – the grammatical, lexical, etc. – competencies of the students but not their ability to become proficient in a natural conversation such that OPI does not inhibit the students’ ability to open, re-open, close, pre-close, establish and change topics, hold and yield the floor, backchannel, interrupt and collaborate, and recognize and produce adjacency pairs which is the purpose of oral communication in the conversational aspect. (Kormos, 1999) Kormos (1999) found the comparison and contrast of the interview and conversation necessary in her study. These similarities and differences contribute in identifying further the validity of either interview or conversation as a measure to assess the proficiency of second language learners or foreign students. Some of these similarities and differences observed by Kormos (1999) are the following: a. Interview is goal-oriented rather than seeking appropriate or proper reaction to different situations. In contrast to conversation as a method of assessing the speaking proficiency of the ESL students, it does not inhibit as much reaction as there is a need for any given situation within a conversation. b. Although conversation is less formal than interview, it encourages a higher level of reactiveness that is an essential aspect of oral communication skills. c. Both methods are on a face-to-face basis that promotes unpredictable outcomes requiring the interviewees to react according to the situation. d. “Interviews can be defined as ‘unequal social encounters in which the interviewer retains most of the rights in the interaction and in which turns are asymmetrically contingent upon each other’” e. “A conversation, however, is ‘an unplanned face-to-face interaction with unpredictable sequence and outcome in which the rights and duties of the interactants are equally distributed and in which speakers’ turns are reactively or mutually contingent’” (Kormos, 1999) Gene Halleck (2005) could not agree more to M. Johnson (2001) when he reviewed the latter’s book entitled The Art of Non-Conversation: A Reexamination of the Validity of the Oral Proficiency Interview regarding OPI. Johnson (as cited by Halleck, 2005) was drawn into a conclusion that OPI, upon the basis of a real-life context of a conversation, is not a valid test for one’s speaking ability. On the other hand, it is recognized as a valid instrument or method in assessing a second learner’s speaking proficiency. Unlike other studies claiming that conversation is a valid instrument in assessing the oral proficiency of second language learners, Halleck (2005) agrees with Johnson’s assessment that it is not an appropriate venue in obtaining one’s oral proficiency. Role-playing, which is a suggested task by Kormos (1999) is just one of the tasks that may be employed in order to provide more rooms for creating a conversation as well as reacting into the responses made by the assessor of a given There are, however, numerous tasks where speaking proficiency of L2 students may be assessed or analyzed. Further studies concerning the assessment tasks in the oral communication language proficiency test have been developed by other language acquisition enthusiasts. Cummings et al (2004) found that the current TOEFL assessment task does not engage the examinees in actually speaking English in any ways. The focus of the tasks were more concentrated on the various forms of the English language which may be easily coached than encouraging them to be involved in a conversation in a proficient manner – which of course requires them still of the various elements from opening a conversation to interrupting one. On the other hand, the new TOEFL assessment task will then address these issues that would encourage the ESL students into a conversation proficiently. (Cumming et al, 2004) In the study conducted by Lynch and McNamara (1998), they have identified two factors affecting the scores of the candidates of second language learners – which are the variability in assessment tasks and the rater judgment. These two factors were said to be the source of measurement error in performance testing. (Lynch & McNamara, 1998) The study conducted by Lynch and McNamara (1998) was basically a response on the study conducted by Bachman et al in 1995 that identified these two factors utilizing the Generalizability theory (G-theory) and Many-facet Rasch measurement as cited in the study. The results from the Lynch and McNamara (1998) showed that the severity between raters creates a significant effect on the candidate’s performance on the given task. Severity of raters pertains to the raters’ level of challenges given to the candidate or the examinee. According to Lynch and McNamara (1998), the success of the candidate depends primarily on the level of challenge that the raters give. These challenges are in other words the tasks that are assigned to them for which the raters would be basing their scores. The more difficult the tasks that are assigned to them, the more that they will find it hard to please the raters, therefore, the lower scores they get. On the other hand, the more easier the tasks become or the more lenient the raters are at giving them challenges, the candidate is also expected to acquire a higher score, thus, the bigger the chances he has of overcoming the tasks easily and to succeed. (Lynch & McNamara, 1998) “Thus, a borderline candidate – that is, one who would have a 50% chance of getting a passing score on an item from the most lenient rater – would have only approximately a 10% chance of achieving the same score from the most severe rater. This is obviously a major cause for concern around the cut-off score in the test.” (Lynch & McNamara, 1998) Based from this study, it could be derived that the scores of the candidates depend on the difficulty of the interviewer – that is represented by the rater in this study. Therefore, the scores of the candidates depend fully on the rater in this case. The inconsistencies of the rater along with their severity are indicative of the variance components for interactions between the facet and the persons or between that facet and other facets. (Lynch & McNamara, 1998) A similar study was conducted and analyzed by Thomas Eckes (2005) through Test of German as a Foreign Language (TestDaF, Test Deutsch als Fremdsprache). Along with the study, Eckes (2005) included gender bias in the study for which he deemed fit to consider in the interaction between the rater and the candidate. The results of Eckes’ (2005) study displayed a similar result showing that severity and leniency of raters greatly affects one’s performance as well as their scores. Furthermore, there had been enough evidence from Eckes’ study showing that the outcome of the candidates’ performances were not affected in any way by gender bias. (Eckes, 2005) Moreover, in the study conducted by Lee (2006), it was found further that an increase in the number of tasks than the number of ratings per speech sample is more efficient in order to maximize the score reliability for speaking. TOEFL has found a new method in assessing speaking or oral communication proficiency of its examinees. Individual or independent tasks and integrated tasks were at the same time implemented as measures in assessing the proficiency level of foreign students in oral communication. Independent tasks are those that require responses from the examinees that would showcase their speaking abilities. Moreover, integrated tasks are those that elicit responses from either the combination of speaking-reading or listening-speaking. Facilitating these tasks elicit the examinees’ capabilities in all form of communication, not limited to speaking alone. The tasks were analyzed according to its dependency on the number of tasks given to the examinees. (Lee, 2006) Results of the study showed that increasing the number of tasks than the number of ratings per speech sample in the speaking or oral examinations for ESL students is more efficient in maximizing the score reliability for speaking. (Lee, 2006) Increasing the number of tasks not only develops a momentum for the students to respond to the various situations but at the same time provides a room for the raters or assessors to analyze the speech samples whether the examinees are proficient oral communicators. In O’Sullivan’s study, however, he measured the differences in performance of the ESL candidates according to their engagement in a conversation with friends and strangers, and that these differences may well be accentuated by the sex of the stranger. (O’Sullivan, 2002) The findings relating to the accuracy of the language are also of interest. Grammatical composition of the spoken language is one factor that should also be considered and measured that is tied up with one’s speaking proficiency. (O’Sullivan, 2002) Since the English grammar has generalized rules, it is easier to measure this aspect of language communication since there is a universal basis for such. This may have been the reason why O’Sullivan (2002) referred to the methods in measuring the grammatical composition language is “infallible”. If the interlocutor or the assessor of the interview is familiar with the rules in the English language, it will be easy for him to measure the technical aspect of oral communication that is represented by the way in which the candidate is able to organize his thoughts using correct grammar usage and sentence composition. Because there had been too much focus on the individual performance of the second language learner candidates, O’Sullivan (2002) studied paired performances among these candidates. Part of O’Sullivan’s hypotheses was that familiarity between the candidate with his partner or with his assessor will obtain higher scores. Being familiar with the person whom one is interacting with will exhibit superior oral communicative performance than those with whom they are not familiar with. (O’Sullivan, 2002) Based from his observations, O’Sullivan’s (2002) observations therefore tell him that this familiarity bridges the gap between the rater and the assessor such that the candidate’s performance, the same time his ability to become a successful communicator, is dependent upon the rater or the assessor. Although Lynch and McNamara’s (1998) studies focused on the severity of the raters, O’Sullivan’s (2002) study is not far too different. O’Sullivan’s (2002) conclusion that the accuracy of speaking proficiency tests is dependent on the interlocutor or the rater is proven to be true. Aside from familiarity between the raters and the candidates or interviewees, there are yet other factors affecting the accuracy of the analyses of speaking proficiencies among English as Second Language students. Sex or gender of the interviewer or interlocutor may affect one’s ability to interact appropriately. (O’Sullivan, 2002) The same observation was recognized by O’Louhlin (2002) wherein interviewers have varying test behaviours. O’Louglin, however focused on the relationship of the gender in the varying responses and interview performance of the candidates. “Such variability includes the amount of support they give to candidates, the amount of rapport they establish with candidates and the extent to which they follow the instructions relevant to their role.” (O’Loughlin, 2002) O’Loughlin (2002) observed from various studies the relationship between language and gender. There had been suggestions from these studies that male and female conversational skills or styles are uniquely distinct from each other. Female conversational styles are recognized to be “collaborative, co-operative, symmetrical and supportive whereas its male equivalent is portrayed as controlling, uncooperative, asymmetrical and unsupportive.” (O’Loughlin, 2002) Finding out whether the earlier research regarding the relationship of gender in the variability of the oral proficiency test behaviours of the raters or interviewers, O’Loughlin (2002) challenges Jennifer Coates’s argument that men and women seem to differ in terms of their communicative competencies in so far as they ‘have different sets of norms for conversational interaction’. (O’Louhlin, 2002) “Coates concludes that ‘women and men may constitute distinct speech communities’. Such claims may have serious implications for language testing since they imply that the construct of communicative competence is not gender neutral.” (O’Loughlin, 2002) Too much generalization about these distinct differences between male and female conversational styles has disregarded other social factors that at the same time affect their conversational styles. Some of these factors that are at the same time significant in one’s ability to interact appropriately and proficiently with others are age, ethnicity, occupation, and sexual identity. (O’Loughlin, 2002) There are, in addition, other factors that according to O’Loughlin (2002) affect the style of men and women’s interaction. Situational factors such as the communicative context also affect one’s ability to interact accurately. (O’Loughlin, 2002) The most recent studies, in contrast to the previous studies that showed distinct differences between the conversational/interaction style of men and women, show that there are also situations where men and women exhibits the same styles of interaction. They are in fact capable of “using a range of conversational styles in different speech contexts. Where men and women exhibit similar conversational behaviour it may be that other aspects of their social identity which override potential gender differences are brought into play. In other words, instead of being fixed, polarized and predictable, the language use of men and women is now seen as varying across cultural, social and situational contexts, sometimes exhibiting difference and other times similarity.” (O’Loughlin, 2002) Thus, using this concluding statements form Coates’ book Women, Men and Language, O’Louhlin (2002) has found that there have no difference between men and women’s ability to respond proficiently in a conversation with ESL students – in this case, the Japanese students. O’Loughlin (2002) also introduces the Minimal responses for which he has patterned and gained from Coates’ book. Examples of minimal responses are ‘yeah’ and ‘mhm’. These responses may not, however, constitute a turn in the conversation process, but ‘they are a way of indicating the listener’s positive attention to the speaker, and thus a way of supporting the speaker in their choice of topic’. And once again, based on previous studies of spoken interaction, Coates claims that women use minimal responses more than men and at more appropriate moments. (O’Loughlin, 2002) On the other hand, O’Loughlin’s study inhibited a contradicting result from Coates’ arguments with minimal responses. “In the IELTS interview data analysed here, minimal responses appeared to serve a consistently supportive function in keeping with Coates’ definition. That is to say, they encouraged the interlocutor to continue speaking by providing a signal to show active listening”. (O’Loughlin, 2002) O’Loughlin (2002) was able to prove that there is no significant relationship or rather a significant effect that gender may have on the performance of the candidate in a speaking proficiency test. Male and female may exhibit a higher degree of variability when it comes to their responses regardless of gender. (O’Loughlin, 2002) Culture, however, may intervene in the accuracy such that socio-cultural norms of interaction in other countries differ from that of the usual process of interaction in the United States or the United Kingdom. However, as much as one on one proficiency interview is given as much attention, Van Moere (2006) devoted his research in studying the validity of a group oral proficiency interview. According to Van Moere (2006) the need for language testers to control the test situation, elicit ratable language samples in allocated segments of time, or create scenarios to force particular language functions may have caused them to overlook opportunities where examinees can be observed producing rich, natural language in conversation with their peers. Sporadic reports over the years agree that the group test format, in which examinees are tested in groups but assessed as individuals, might be appropriate for certain test situations, or as part of an oral test battery. (Van Moere, 2006) Group oral test is, according to Van Moere (2006), is one of the most reliable measure in assessing one’s ability to interact with his fellow candidates. One of its advantages is that it is resource-economical such that one rater could test and rate up to five or six examinees in just one session. There is going to be need for fewer assessors or raters to be able to test hundreds of candidates especially in schools and universities. There is at the same time lesser intervention from the examiners or the facilitators of the test such that the examinees or the candidates will become more aware of their fellow candidates and their strategies or styles as to how they will be able to interact appropriately with them. A feeling of intimidation may not totally be avoided since they have the idea that they are being observed, but it is lessened in such a way that they will be able to focus more on their performance. (Van Moere, 2006) A positive washback of influence may also result from the said task especially inside a classroom where there is a greater needs to practice the communication skills, particularly the speaking or oral communication skills of the students or participants with diverse cultural backgrounds. The participants in the group discussion will be the ones to create various situations and topics of conversation that would come naturally within the participants that would encourage collaboration and cooperation from each other. They are given the opportunity to take control of the situation for which they will be required to respond to each participant according to the type and topic being discussed. Furthermore, this does not alone involve the speaking process, but they are as well practiced to exercise their listening skill which is one of the most important factor upon the achievement of a superior conversational skill. When one does not know how to listen, he will not be able to communicate or converse appropriately. As a result, learning therefore takes place naturally for the participants. (Van Moere, 2006) Conclusion Several factors are affecting the Oral Proficiency Interview in assessing the speaking abilities of L2 candidates. This includes the rater’s judgement or bias on the person being interviewed, as well as the number and the quality of task assessments given to the candidates which affects them in response to the severity of the task given to them. However, arguments on the number or the volume of tasks given to the candidates in the assessing their ability to perform such tasks continuously seek for more research as there have been contrasting results from the various research studied. On the other hand, studies also prove that the interviewer’s gender has nothing to do with the performance of the candidates. No significant result was found when men are interviewed by women, and vice versa. Furthermore, other alternatives should be explored in assessing the proficiency level of the candidates such that they may be found more valid and reliable. One of which is the simulation or the role-playing method for which the candidates are being elicited to have open the conversation, close it when deemed necessary, create a feedback or even interrupt the other person speaking. In addition, the use of an oral group interview may also seem appropriate in assessing the candidates’ proficiency such that it inhibit a more natural conversational situation among the members of the group allowing the to converse liberally. Since speaking or oral communication is the most common form of communication that is utilized in all aspects of human interaction, its assessment should be valid as well as reliable. Works Cited: Brown, A. (2003) Interviewer variation and the co-construction of speaking proficiency. Language Testing 20 (1). Cumming, A. et al (2004) A teacher-verification study of speaking and writing prototype tasks for a new TOEFL. Language Testing 21 (2). Eckes, T. (2005) Examining Rater Effects in TestDaF Writing and Speaking Performance Assessments: A Many-Facet Rasch Analysis. Language Assessment Quarterly (2) 3. Elder, C. et al (2002) Estimating the difficulty of oral proficiency tasks: what does the test-taker have to offer? Language Testing (19) 4. Halleck, G. (2005) Unsubstantiated Claims About the Oral Proficiency Interview (A Book Review) Language Assessment Quarterly (2) 4 Kormos, J. (1999) Simulating conversations in oral proficiency assessment: a conversation analysis of role plays and non-scripted interviews in language exams. Language Testing (16) 2. Lee. Y. (2006) Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks. Language Testing. (23) 2. Lynch, B & McNamara, T. (1998) Using G-theory and Many-facet Rasch measurement in the development of performance assessments of the ESL speaking skills of immigrants. Language Testing (15) 2. O’Loughlin, K. (2002) The impact of gender in oral proficiency testing. Language Testing (19) 2. O’Sullivan, B. (2002) Learner acquaintanceship and oral proficiency test pair-task performance. Language Testing (19) 2. Sanchez, L. (2006) Bilingualism/Second-Language Research and the Assessment of Oral Proficiency in Minority Bilingual Children. Language Assessment Quarterly. (3) 2. Van Moer, A. (2006) Validity evidence in a university group oral test. Language Testing (23) 4. Read More