Avatar-Based Fingerspelling Practice Software - Arabic Sign Language - Literature review Example

AVATAR-BASED FINGERSPELLING PRACTICE SOFTWARE: ARABIC SIGN LANGUAGE School of Computing Arabic Sign Language (ArSL) is the primary language used by the Deaf Community in Saudi Arabia. It is a visual-gestural language containing hand, facial, and body movements. Fingerspelling is an essential part of ArSL and is a necessary skill for communication that is used for acronyms, technical terms and words from foreign languages. A common difficulty found within the Deaf community is the lack of resources and tools to learn ArSL. In Saudi Arabia, the software and methods of communication among the Deaf are still in their infancy. A tool featuring a fingerspelling avatar, an animated three-dimensional human-model, will benefit the Deaf community of Saudi Arabia by portraying the Arabic language and deaf students will learn by practicing with the avatar. There are currently two existing tools for learning Arabic finger-spelling. Both methods are incomplete and unrealistic due to the absence of transitions between letters. Our proposed tool will provide the deaf with a natural, extensible and appealing learning experience as opposed to the rigidity of video recordings and unconnected signs. The design of the tool puts into account the transitioning of the letters to make the users better understand and recognize the finger-spelling. We will also implement options for viewing the animation at different speeds and angles, as well as allowing users to type what words to display. User tests will be conducted to determine the effectiveness of the tool, as well as evaluate it on realism, recognizability, and comprehensibility. Keywords: Animation, Arabic Sign Language, Finger-spelling, Computer Graphics, Deaf Education Introduction and Overview Making learning easy for the hearing impaired is an important matter that needs significant consideration. Deaf Arabs have been sidelined for a long time in terms of education and other opportunities for jobs in the marketplace. Communication is a problem that has been deeply entrenched in the society concerning the Deaf. Those who hear have a hard time understanding the Deaf especially in Saudi Arabia. The hearing and hearing impaired do not have a means of learning that can help them communicate effectively with the deaf. Arabic Sign Language (ArSL) is the natural language used by the Deaf Community in Saudi Arabia. It uses hand signs, finger-spelling, facial expressions, and body movements. Fingerspelling includes a symbol to represent each letter in a written alphabet. Fingerspelling is an important part of a meaningful conversation, which is used for proper nouns, technical terms, acronyms, and words from foreign languages. Wilcox studied hearing students learning American Sign Language students and found out that they had a great difficulty in reading fingerspelling words (Wilcox 1992). To learn the static fingerspelling symbols is not hard but to master the reception of finger-spelled words is often reported as the most difficult skill to master because finger-spelled words contain many symbols that are displayed in rapid succession (Jamrozik 2010). Use of technology in sign communication has increased rapidly in the last few years. A digital personal tutor, avatar, system in which, the system generates animations of ASL via a three-dimensional human-model. As such the introduction of avatar-based learning technologies is a significant step in redeeming the situation since those who hear will have a better understanding and appreciate the deaf. They will be able to learn along with the deaf by using the learning / practice tool. Thus, communication will improve, and the deaf will be able to educate themselves on their avatar-based software without much supervision. The deaf will have a tutor at hand and thus have the opportunity to practice more than they would with their interpreters or teachers. People with hearing impairments have a difficult time interacting with the society since some of them may not know how to communicate through sign language. There are many educational materials in English language as compared to those in Arabic. The use of an avatar practice tool will help the deaf students learn quickly. The visualization of the software and the consequent interaction with the avatar will make use fingerspelling that is another way of learning sign language. The students will be provided with many types of activities and tests to increase their understanding of finger-spelling. The sign language tool will use the Arabic language in the software. The avatar based software will use finger spelling that the learners will have to learn when interacting with the software. Fingerspelling is not hard to learn but mastering it is the issue. The animated hands will be more pronounced to ensure that the users have a clear and vivid picture of the signs in the Arabic language. As a result, the fingerspelling will help in understanding various sentences constructed in the Arabic sign language. The software will thus include an animation and various buttons that will be used in operating the software. The interface is quite simple to prevent difficulty that may arise in its usage. The software design will include all the fingerspelling techniques in similar software but in the Arabic version. The learners will be able to learn and form sentences using the software and in the end master the fingerspelling technique. As a result, the deaf will have the convenience of learning the fingerspelling more quickly since they will not have to wait for their tutors or interpreters but will be able to master the fingerspelling techniques on their own. Usability Concerns The use of the software is a crucial concern for the Arabic sign language users. The avatar-based software will be an individual tool that the users will use to learn sign language. The users will have to practice a lot on their software and ensure that they master the various fingerspells to ensure their accuracy in communicating correctly. The users should not take the avatar-based software as a game but as a learning tool. As a result, the more they practice them more they will be acquainted with the sign language. Technical Concerns The process of coming up with ASL handshapes is a challenge. The appearance of the hand in the software is in a bent position. The handshape to be used in the software is made so that it is as natural as possible and, as a result, one can understand the signs shown quickly. Various challenges arise especially where fingers collide. In some cases, the letters may lead to the thumb being hidden since the hand is tightly held to show such letters. Therefore the more natural the hand animation, the more subtle the problems that would arise due to collision of fingers Related Work/Literature Review Various studies on fingerspelling have shown difficulty in the learning of fingerspelling among students (Hernandez, 1997). Learning the technique of fingerspelling is not difficult but mastering it is hard because of the many hand symbols that are involved (Robison & Jensema, 1996). Fingerspelling represents the letters of the alphabet and helps the deaf to understand the language through expressing and practicing the language. The American Sign Language has lesser use of symbols than fingerspelling does (Vogler & Metaxas, 2001). The unavailability of interpreters to help the deaf in learning English through sign language makes communication more difficult. The lack of interpreter has helped many researchers to come up with a constant companion that can assist the deaf in studying sign language together with the alphabet in the English language. The various systems have been introduced to enable the deaf to have ample time in learning fingerspelling to understand the English language (Ghanem & Albidewi, 2013). The users can, therefore, use the various fingerspelling in the software that will help in learning the different symbols and improve the ease of mastering fingerspelling. EL-Bendary et al. (2010) notes on the translation of gestures automatically from the manual alphabet to Arabic sign language (El-Bendary, 2010). The method uses the various images of hands in the interpretation to sign languages (El-Bendary, 2010). The user has a natural interface, which he or she can associate with more easily. Studies have shown the ArSLAT has an accuracy of up to 91.3% in recognizing the letters of the Arabic alphabet. As a result, Arabic sign language through fingerspelling and the use of relevant software can be achieved. The use of software provides an environment where the user has countless opportunities to practice constantly. The tool in the study will help in improving the communication significantly among the hearing impaired by merging the use of avatars and fingerspelling clearly teach the hearing-impaired. The hearing and impaired people have a chance of considerably learning and appreciating each other by using the Tawasol App. Tawasol is an app developed in conjunction with Software and Knowledge Engineering Research Group and the charitable foundation of Saudi Arabia. The foundation is the Princess ALanoud Foundation. The hearing and impaired people have a chance of significantly learning and appreciating each other by using the Tawasol App. Tawasol is an app developed in conjunction with Software and Knowledge Engineering Research Group and the charitable foundation of Saudi Arabia. The foundation is the Princess ALanoud Foundation. Features The characteristics of the app are many and include among others finger-spelling editor, tutorials and a dictionary. The dictionary consists of the various type of words in Arabic. The different types of words form the basis of selection. Users choose a certain word, and a video clip that has the sign of the word chosen appears together with the interpretation. The dictionary has the search function, which enables users to look for particular words. It has the repeat feature, which determines the frequency of looking up signs. The favorite feature helps the user save favorite words that he can watch later while Sound control provides users with an audible speech of the various words. The tutorial function provides users with tests. The tests have different levels whereby users can progress as they match the multiple video clips with signs to the words given. Another feature of the tutorials is that learners can choose the easy or hard option for the different levels. As one progresses, the tests become more challenging. The fingerspelling app displays the phrases entered by users into sign language. The app consequently shows the symbols for every letter of the expression entered. The alphabet used contains 3D Arabic sing language that introduces learners to the Arabic version of sign language. It has numerous questions with each sign having a 3D navigation. The tool does not teach spelling of words but uses practice where the users learn the Arabic alphabet. The American Sign Language tool has been a necessity for the hearing and hearing impaired especially in learning fingerspelling about the American Sign Language. The Arabic manual alphabet differs significantly from the manual alphabet of American Sign Language. The study will design software in the Arabic language that will draw on lessons learned in developing the American software, which is in English (Wolfe 2006). The tool used in America “Fingerspelling Tutor,” is part of the development in the use of American Sign Language. The design of the software puts into account the transitioning of the letters to make the learners understand better and recognize the finger-spelling. The software is interactive, and the user gets to practice and understand the different fingerspelled words. The various transitions in the displays help the learner to learn words that are produced in succession. Research Design and Methodology The team is working on the avatar based Arabic software using fingerspelling concentrated on the prototyping of fingerspelling. As a result, the problem of coming up with hand animations would be likened to those of the different fingerspelling for each word. The hand animations would represent the different fingerspelling together with associated letters and words in the Arabic sign language. User feedback is critical, and various feedback mechanisms of the software would be present that will help improve the software in the short and long term. Speed in the software presentation of the different fingerspell word is a paramount concern of usability. The efficiency of use of the tool is thus an important part of the project. The users, therefore, have to conversant with the utilization of the software through the interface that is user-friendly. The usability will be determined by how easy the users get to work with the fingerspelling tool. Usability tests were conducted to determine how the various participants in the project fared on with the different hand animations and subsequent words in the software. The users were, therefore, to give feedback on the hand animations in tool especially the multiple images together with the presentation speed of the animations. The participants in the study were from elementary school together with those at the deaf expo that explores the various issues that affect the hearing impaired. The members who took part were either a bit proficient or more proficient in Arabic sign language. Every person who participated was shown various CG animations at three different speeds. The highest was first. Participants were required to identify the finger-spelled words presented at the three different speeds. The various members were gauged on the speeds they were able to spell the phrase with the initial presentation having the highest rate. Different words were repeated for each of the participants. The deaf participants from high school were passed through various tests involving animations and images. A large part of the students could tell the words finger-spelled at the most top speed in the first and subsequent presentation. For the Deaf Expo, every second those participants averaged 2.5 words. The figure concurs with various studies concerning fingerspelling rates. The representation of the tool should be flexible especially in algorithmically changing the various vocabularies that are part of the Arabic sign language grammar. The merging of the fingerspelling animations together with the video utterances should concur, and the animations should be more real as a result. As a consequence, animations shown by the avatar in the tool should be as efficient as the use of a video showing a human sign language interpreter. The analogy of modeling the fingerspelling hand animation follows various evidential appearances of animations in video games whereby characters are represented as nearly identical to humans. Sign language has undergone many changes due to the research done in the linguistic field especially by Stockoe, who found out that gestures or visuals systems would be used for communication. Thus, the discovery of the use of gestures was a welcome advancement in communication among the deaf. The deaf, as a result, have a conventional sing language that they can use in learning various languages if possible and communicating with other people in their countries or throughout the world as a result. The inclusion of fingerspelling is thus an important invention that goes a long way in helping the hearing and hearing impaired to learn and appreciate the sing language especially in Saudi Arabia. Plan of Work and Outcomes (deliverables, schedule) The language of the proposed software is Arabic, and the finger-spelling and the software interface will be in Arabic. The fingerspelling software consists of an avatar that is either male or female. The student can choose the desired avatar from the preference section that guides the learner. The avatar will tutor the student in finger-spelling. The client operated software will have various levels that take the user to different lessons. The levels will consist of levels that will explore the alphabet, words, and sentences progressively. The user will learn at his or her pace, which would not be the case in a classroom setting. The alphabet level acquaints the user with a chart that has the manual alphabet. The users can have practice lessons and quizzes regarding the letters of the alphabet in the chart. In the Beginner level, users will have the opportunity to practice discovering two and three-letter words using multiple choices. The buttons will be evident in the proceeding levels. At the intermediate level, the users will choose a category where they get to select from similar answers to identify a finger-spelled word. The advanced session is more complicated than the other levels and requires the user to complete sentences. The 3D animation feature is a significant breakthrough that will help the learner to view the hand wholly from every side. As such, the user will be able to use his hand to turn the animation on every side to get a clear and vivid picture. The software will be customized to be used on different platforms like smartphones, tablets, IPad, and computers. The game option will be included in the software later. Conclusions and Future Work Fingerspelling and the use of software that can be employed by the deaf is an important advancement that will improve their communication and the mastering of fingerspelling, which is an important element sign language. As such, the use of software provides a teaching environment with the software taking an important and present tutorial role. Future work should concentrate on the switching to different languages by the integration of various languages. The software can, therefore, have the Arabic sign languages and American sign languages to enable the users to have proficiency in the both the sign languages. The clarity of the animations should further be concentrated upon to provide a clear picture of the fingers and thumb. References Ghanem, Sakher, and Ibrahim Albidewi. "An Avatar Based Natural Arabic Sign Language Generation System for Deaf People." Computer Engineering and Intelligent Systems 4, no. 11 (2013): 1-6. Halawani, Sami M., Daut Daman, Sarudin Kari, and Ab Rahman Ahmad. "An Avatar Based Translation System from Arabic Speech to Arabic Sign Language for Deaf People." International Journal of Computer Science & Network Security 13 (2013): 43-52. Robison, Joseph, and Carl Jensema. "Computer speech recognition as an assistive device for deaf and hard of hearing people." DOCUMENT RESUME 948 (1996): 154. Vogler, Christian, and Dimitris Metaxas. "A framework for recognizing the simultaneous aspects of american sign language." Computer Vision and Image Understanding 81, no. 3 (2001): 358-384. Hernandez, Ricardo Alberto. "Teaching fingerspelling: The role of transfer of learning strategies, serial ability, and self-efficacy." (1997). El-Bendary, Nashwa, Hossam M. Zawbaa, Mahmoud S. Daoud, and K. Nakamatsu. "Arslat: Arabic sign language alphabets translator." In Computer Information Systems and Industrial Management Applications (CISIM), 2010 International Conference on, pp. 590-595. IEEE, 2010. Wolfe, R., Alba, N., Billups, S., Davidson, M. J. … & Young, J. (2006) An Improved Tool for Fingerspelling Recognition. Technology and Persons with Disabilities Conference 2006. California State University at Northridge, Los Angeles, CA March 20-25, 2006. Wilcox, S. (1992). The phonetics of fingerspelling. Philadelphia/Amsterdam: John Benjamins. Jamrozik, Diana Gorman, Mary Jo Davidson, John C. McDonald, and Rosalee Wolfe. "Teaching Students to Decipher Fingerspelling through Context: A New Pedagogical Approach." In Putting the Pieces Together: A Collaborative Approach to Educational Excellence-Proceedings of the 17th National Convention of Interpreter Trainers, San Antonio, USA, pp. 35-47. 2010. Budget Budget Item Amount Total a Personnel Summer Stipend for Graduate Research 5000 b Direct Expenses Arabic Sign Language interpreter 1920 c Participant Incentives $10 gift card for participants ( 50) participants 500 d Travel International Travel 1500 Total 8740 Read More