Human Personalities in Usability Tests - Research Paper Example

Human Personalities IN Usability Tests I. Introduction The Big Five Human Personalities According to Psychologist, there are Big Five distinct human personalities. Different psychologists call them with different names, but their common denominator is the traits that are exhibited by each personality type. In a nut shell, they include:- Openness – People who appreciate art and imagination of new things in life, curious, love adventure open to new experiences. Conscientiousness – Very organized people with a strong sense for duty. Usually achievement-focused, disciplined and dependable. Agreeableness – People who easily trust depending on the kindness and warmth of other people around them. Neuroticism – They worry about everything and are usually very anxious. Introverts/Extroverts – Most common people personalities known. Extroverts are very sociable, talkative and draw energy from multitudes. Commonly known as “Social Butterflies” The introverts are a complete opposite of the extroverts. Usability Testing This refers to a technique used in a user centred interface system to evaluate if the intended task will be carried out satisfactorily, effectively and efficiently. – Gerry Gaffney Information & Design 1999 1.1 Background Different human personalities reflect different results in the usability testing. It is very important to have an in depth psychological understanding on human personalities even as one conducts these tests. The choice of participants is widely believed to influence the results of the usability test. For this study, it is postulated that there is high likelihood of the participant’s personality to influence the overall usability testing experience. The ease of use of a product by a consumer can be quantified using usability tests. Usability requires a thorough understanding of the expected users of that product, their need, and the circumstances or environment of the product usage. The as human centred design approach of efficiency, effectiveness and satisfaction comprises the overall design process of a usability test. Precisely, the personality dimensions of extraversion-introversion have been claimed to have effect on user’s interaction with a computing product. Therefore, it is worthwhile for this study to establish the influence of the personality dimension on usability testing experience. 1.2 Statement of the problem Usability tests are motivated by a companies’ need to increase revenue and fulfil their objectives and goals. A user testing procedure is a social encounter between both the product and the participants or the evaluator and the product. As a result, the participants’ personality tends to have considerable effect on the overall knowledge on the product experience. After understanding the participants’ personality, it is then possible to analyse results from the usability tests and utilise them in developing or improving the product. Furthermore, personality information is typically necessary in developing a product since it helps the manufacturer to incorporate the wide range user needs of individuals (Hertzum et al., 2009). It has been shown that personality has substantial effect on a user’s interaction with a product. Previous researches have established extraversion and introversions dimensions of personality as most significant in influencing usability tests (Barendregtet al., 2007; Hertzumet al., 2009). A major drawback in development of efficient interfaces on product use is that there has been limited empirical evidence on the effect of participant personality on the usability testing (Lindgaard and Chattratichart, 2007). This research is interested in extraversion and introversion personality dimensions from the view of how the users interact with a product during testing, and ability to describe their experiences. The results will be of significant importance in adding information to existing literature on usability tests as well providing valuable insights on relationship between extraversion and introversion personality dimensions and user experience to product developers and marketers. 1.3 The general objective of the research Overall Objective The main aim of this study is to test how major human personalities can affect the outcome of usability testing. Specific Objective To be able to achieve the aims of this study, the following objectives must be accomplished: To establish in depth relationship between extraversion personality dimension and the outcome effect on usability testing. To assess the difference between extraversion and introversion personality dimensions in usability testing. 1.4 Research questions How does the relationship of human personalities of an introvert and an extrovert affect the outcome of the usability test? 1.4 Overview of research methodology The overview of the proposed research methods can be summarised in the figure below. Figure 1: research diagram Objective From figure one above, the test object or targeted website is the University website. The overall objectives of the study should be clearly outlined and communicated to the people taking part in this study. These include those that will be recruited as participating and those who are conducting the tests. The specific objectives may remain with the “crew” or those carrying out the tests. Hypothesis From the book knowledge that is extracted from the references about the personalities and the main characteristics they exhibit, a natural phenomenal will be explained and propositions of the possible outcomes pin pointed. The documentation of which will form the literature review. Recruitment of Participants The tasks designs will be based on the information of interest, which is the content on admissions to the university. The selection of the participants will be based on the initial tests for degree of extraversion personality dimension using the Myers-Briggs Type Indicator. Thereafter, the web usability test description and recruiting screener exercise will be conducted to ensure that the participants are suitable for the study. Those participants who qualify will be given consent forms to review their willingness to further continue with the study. Selecting the Tests Object & Designing the Scenarios and Tasks The tests objects and the Designing Scenarios and Tasks which are well prepared upfront and well documented on cards should be distributed to the consented participating recruits beforehand. These clarify their input requirements and erase any anxieties or uncertainties that might affect the outcome. Pilot Testing & Actual Testing Pilot testing of the tasks will be conducted whereby the methods will be tested and the average time taken for the completion of the tasks to be established. This provides an opportunity of dry runs and opens up for more fine tuning before the actual test. It also creates and allows the participants to do the actual tests with a lot of ease. Data Collection & Data Analysis Data collection in this study will be done in different ways and forms. These include pre-printed forms or cards and questionnaires, video cameras, audio recording. The data analysis process will therefore be involving and very tedious. The data analysis crew should therefore be prepared upfront on their task and be well briefed on the ultimate importance of the analysis. Anything short from the excellent data collection and analysis should not be accepted because this is a very important point of the project. Not that the other section are not import, but a lot of emphasis should be put in this section. The team in this section should be invested upon heavily in terms of time, equipment and motivation to achieve out intended objectives effectively. Results Reports and Conclusion From the collected data and analysis, the reports will be derived. The data collection crew as had been prepared early, we expect an excellent data from which crucial information will be derived. Conclusion will be drawn from whatever information that will be obtained from the reports. 1.6 Challenges of research The potential challenges for this research includes recruiting the right participants of the study. The right participants need to be recruited to ensure that desirable quality of the tests results is achieved. For this study, extravert and introvert individuals should be recruited for the study hence prior demographic and attribute checking is necessary to ensure participants match the user well. Another challenge is distortion of information especially when completing the tasks. The participants in the usability testing exercise may lose concentration and pay less attention to the activities. To mitigate this challenge, the evaluator in the study will ensure that the participants are involved at every step of the exercise especially using visual aids and giving detailed summary what is expected of them. Additionally, designing the right tasks will pose challenges for this study. However, with prior identification of conditions and events that could motivate the participants to use the test object as well as constant exploration of the context of use of the test object, the challenges can be alleviated. 1.7 Structure of research Chapter one of the researches gives the background information on the usability testing. It also outlines the factors including personality that influence the outcome of usability test. The motivation behind the study and the aims of the study are also outline in this section. Chapter two of this research shows review of existing literature on usability testing as well as evaluation methods for usability tests. It also elaborates vies of other scholars on the factors affecting usability tests as well as candidate methods for usability testing in this study. Chapter three outlines the proposed methods for usability testing which include description of the test object, participants of the study, tasks and usability measures. It also outlines ethical considerations as well as potential risks of the study. Chapter four shows all the data collection and analysis methods crew, equipment and what is expected of the at every stage. The result and reports will also feature in this section. Then a conclusion will come thereafter but not as a chapter but as a brief paragraph. II. LITERATURE REVIEW 2.1 The Definition of Usability Bevan (2009) defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”. Efficiency is the extent to which scarce resources like time, effort and costs are well used for the intended undertaking. Effectiveness is the degree to which a person is successful in coming up with the desired results. On the other hand, satisfaction is the fulfilment of a person’s expectation in a task. According to Dumas & Redish (1993), four signs are good indicators of low usability levels of a product. They include the need to rework low level of product usage, usage experience dissatisfaction and work around presence during the product usage. Usability can be defined from the operational point of view. The first aspect in this definition is the learning ability. This term gives a clear description on how fast a novice user can advance to a basic proficiency to an advanced level and generate the required result at each level. Users have been known to prefer products that are easy to learn and produce the desired results after a little training period. The second aspect is memorability, which refers to the ease of use for an occasional user in utilising a product after a long time without using it. Consumers prefer a product with memorable usability tan the one with unmemorable usability. The third aspect is error tolerability, which is the extent to which a product can allow a user to correct or recover from their errors. Products with high error tolerability are preferred since the user can recover easily and the user does not always have to start the procedure all over again. The last aspect is efficiency, which is quantified, by how productive a competent user can be with the product. Users desire high-level efficiency for their high quantity and quality of their productivity (Dumas &Redish 1993). 2.2 Usability Evaluation Methods (UEMs) Battleson, Booth and Weintrop (2001) stated that there are many types of evaluation usability. Methods in software systems, but generally there are three main types namely, inquiry methods, inspection methods and formal methods. According to International Standards Organization (ISO) of defining usability. Powell suggests the definition of website usability as "the extent to which a site can be used by a specified group of users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use" (Whitehead, 2006). 2.1. Inquiry Methods: Holzinger (2005) explains these methods of evaluation usability as representative users and provide information about how can users solve the problems in a particular software product by using their experience. It also has been known as test methods. There are several methods for evaluating these methods such as thinking aloud, field observation and questionnaires. 2.1.1. Thinking Aloud (THA): Nielsen, Clemmensen and Yssing (2002) comment that thinking aloud is one of the most common methods in usability evaluation and it tends to discover design problems on user interface of software system. It is possible that THA is the most used method in inquiry evaluation methods because it asks end users to think aloud while using the system. Thus, as Holzinger (2005) suggested this leads to facilitate to identify the main errors of the systems. The benefit of THA is an ability to help users to find problems on the system while performing the tasks. In contrast, the disadvantage of THA is often time consuming as it demands bringing representative users into the testing place (Holzinger, 2005). 2.1.2. Field observation: Hom (1998) defines field observation as the best technique to decide the usability requirements of user. The first kind of this observation is visiting at least one user in his work lab and often asking questions of the users to understand how the users are using the system to achieve his aims. Another way of this observation is use indirect way to ask the user such as using video, but it is not common method. In addition, data logging is another means of field observation which can provide information and statistics about the field frequency (Holzinger, 2005). 2.1.3. Questionnaires: George (2008) describes questionnaires as collecting information of users needs by asking them questions and it can be managed online or on paper. Holzinger (2005) comments that although questionnaires need experience to design, they are often very useful for testing the satisfaction of users, especially the product of use. There are two common types of questionnaires which are an interview and indirect methods. Indirect method can explore the active interface of a user by collecting users’ opinion about that interface. This method has some drawbacks such as it needs a large number of participants to evaluate the process to be useful as well as it identifies a small number of problems than the other methods. Interview technique is a simple type of questionnaire, it could be adapt with the user reaction and encourage the situation. The advantages of this type of evaluation methods include that they reflect personal user priority, satisfaction and concerns can be identify in a simple way, thus statistics were collected by questionnaire (Holzinger, 2005). 2.2. Inspection Methods: Nielsen and Phillips (1993) argued that inspection method is a group of techniques that are depended on evaluating the interface by experts. It were used by developers to expect the problems of usability which might be discover by users examining (Matera , Rizzo and Carughi 2006). They suggest that the most common types of this method namely, heuristic evaluation and cognitive walkthrough. 2.2.1. Heuristic evaluation: Matera , Rizzo and Carughi (2006) comments that heuristic evaluation is the most popular informal technique of inspection methods and it knows as discount usability methods. It needs up to five experts reviewers to test the usability methods in interface of the page or website which are prototype, set of heuristics, set of tasks, consistency and evaluation form (George, 2008). He features it can occur at any period of the development life cycle and it may take 12 hours to a week to complete depending on some issues such as the website depth and the kind of evaluation. However, it is quick, effective, economical and convenient, but it dependent on understanding and experience of experts so, there is high possibility of making errors (George, 2008). 2.2.2. Cognitive walkthrough: Whitehead (2006) explains cognitive walkthrough as asking experts to find problems while achieving the user tasks. In this method the evaluators begin with analyse the specific task of steps required by users to complete (Matera , Rizzo and Carughi 2006). Then experts walk through steps to test the system interface by asking themselves some questions at each stage to achieve the aim of users. For example, 1.Will the user observe that the right action is available? 2.Will the user attempt to achieve the correct effect? 3. Will the user receive finding? (Matera , Rizzo and Carughi 2006). As a result, if the examiner believes that "these questions cannot be answered satisfactorily, then a potential interface problem has been located" then he notes it and try to find a suitable solution for it. 2.3. Formal Methods: Battleson et al., (2001) define formal methods as observing users while using a website or prototype to execute given function or obtain a specific aims. A good example of formal methods is the GOMS model (Goals, Operators, Methods and Selection rules) which designed to speculate the expert user effectiveness (Nielsen and Phillips, 1993). Battleson et al, (2001) describe the results from this type of methods are useful for website interfaces and it gives designers more efficiency in the design stage because it changes the opinion with dependent information on users. The main drawback of this method is that it does not find usability problems in the evaluated systems. 2.3 Factors effecting on usability testing Literature suggests many factors that affect usability testing results negatively or positively. The first one is usability testing tasks. These are a representation of users’ achievements after using a product. Choosing a particular wrong task can lead to numerous complaints. Various types of tasks used in usability testing include structured task, which entails guiding users systematically on what to do to identify potential problems. In addition, uncertain tasks where users are uncertain on the accomplishment of their effort can be used. Problem-solving task where tasks can also be constructed in form of a problematic statement, and then administered to the users. They can then be observed on how they would potentially behave in reality when using a product (Lindgaard & Chattratichart, 2008).Environment can also affect usability test results. This is possible especially in computing environment. Lindgaard & Chattratichart, (2008) suggest that every participant in usability test should be in identical environment with the other users. Characteristics of the user can affect the test results, thus for the best results the users participating in a usability testing study have to reflect the real user of a product otherwise the problem with a product will not be identified. Also of concern is personality of the users under the usability testing. Several researchers have investigated the role of participant personality on usability testing especially using the Myers-Briggs test to measure degree of personality (Capretz, 2003; Molichet al., 1999; Rutherfoord, 2001; Wixon, 2003). Personality of the recruited participants has been shown to have a clear effect on interaction of a user with products under usability tests thus significantly determine the success of the usability testing procedures (Capretz, 2003). Capretz (2003) suggests further that it is always desirable to recruit participants who will expose a wide range of problems in usability of a product. Influence of extraversion and introversion personality dimensions has been documented widely in modern psychology as the main aspects influencing interaction between users and products under usability tests. Burnett and Ditsikas (2006) investigated influence of extraversion and introversion on usability tests based on a range of tasks for an e-commerce web site and found out that introverts uncovered less usability problems than extraverts do. Burnett and Ditsikas also noted that it took longer time for extraverts to complete the usability testing sessions compared to the introverts. Capretz (2003) also found out that extraverts have high levels of motivations when participating in usability tests thus discovered more usability problems on the product under test compared to their introverts’ counterparts who provided less feedback during the usability testing sessions. III interactive. RESEARCH METHODOLOGY 3.2 research method This research is based on the experimental approach as this approach is known to be very effective in terms of achieving the research aim and objectives and answering its questions mentioned previously. This research project will also involve quantitative and qualitative measurements. The research questions require the use of between-subject design in this study. This design is very useful in preventing learning effect between compared conditions. 3.3 Test object The Newcastle University library website will be targeted as the test object in this study. The rationale for using this product stems mainly from four reasons: first, it is a dynamic website with multiple interactive features and functions. Second, the website is manageable in size, that is, the numbers of pages are considerably few to allow for thorough evaluation of its usability level. Third, representative users of the site are easily accessible as they are mainly university students. Fourth, the site possessed a number of clear usability problems. This was determined using heuristic evaluation conducted by the author. 3.4 Task Tasks will be designed to effectively reveal problematic features of the website. The participants will be required to perform four tasks within a 35 minutes session. The ‘think aloud’ technique will be employed in all the activities. The participants will be required to begin at the homepage of the website then navigate through the website without attempting to accomplish any particular task. This activity will allow the participants to simply gather a general impression of the university using the website. Ideally, the participants will be allowed to navigate at least five to ten pages of the website for approximately one to two minutes. Thereafter, the task 1 (appendix) questions will be administered to the participants. The second task would entail navigating to the section of the website relating to particular postgraduate and undergraduate degrees. The evaluator will choose the section of the website for this task for each participant based on the participant’s appropriate category (undergraduate or postgraduate). For this case, information regarding a particular degree will be required to be obtained by the participant. The number of clicks and time taken to navigate to the exact information will be recorded. The guiding questions for this activity are found in task 2 (appendix). The third task is aimed at testing the participant’s abilities in locating admissions applications information. They will be required to navigate from the home page to the particular section for either postgraduate or undergraduate admission as per evaluator’s instruction. The participants are expected to find out information on application deadlines. The guiding questions for this task are found in Task 3 (appendix). The last task will be to locate the financial requirements for the particular degree the participant was assigned. The participant will be observed on the ease of retrieving information regarding the tuition and other costs in the first year of study for the particular degree course. The guiding questions will be provided in task 4 (appendix). 3.5 Participants A desirable random sample of participants will be drawn from the population of users of the selected university website. Ten participants will be considered for the user testing exercise. In particular, the personality of the participants will be established using the online Myers-Briggs Type Indicator. As such, five participants with high degree of extraversion (extraverts) and five participants with low degree of extraversion (introverts) will be chosen. The gender of the participant as well as their ages will determine their suitability for the usability testing exercise. Furthermore, the participants will be given some instructions on the general objective of the study. The participants will be provided with overview of the ‘think aloud’ technique, which is a popular usability evaluation method (Peuteet al., 2015). The technique is designed such that it encourages the participants to converse on what exactly they are attempting to do while using the website, and to examine their failures and successes. Once the evaluator is certain that the participants are confidently “speaking aloud” then scenario-based tasks can be administered to the participants (Peuteet al., 2013). 3.7 Data analysis Statistical analysis of collected quantitative data will be conducted using IBM SPSS statistical software package (version 20). The relationship between personality and usability of the test product of interest will be assessed by Pearson’s correlation coefficients. The number of problems for this study depends on the number of tasks. Generally, the ease of navigation, appearance of the pages and availability of information in the website will determine the users’ experience about the website. As such, the identification of the usability problems can be made for the University website. For this study, the task performance will be measured based on time taken to complete particular task. Furthermore, the number of clicks and path of navigation when retrieving the required information will be recorded. The ease of navigation and the few number of clicks will be indicative user’s identifying fewer usability problems and vice versa. The extent to which users have attained their practical goals can be used to evaluate user satisfaction. Likability of the website, that is the satisfaction based on participant’s perception of achieved pragmatic goals. Also satisfaction of the website will be measured based on pleasure, which is the emotional response towards the website. It will also entail measuring satisfaction based on physical comfort when using the website. System Usability Scale (SUS) questionnaire will be administered at the end of the session as it is considered a standardised and dependable subjective assessment method for testing usability of a system (Kortumand Bangor, 2013). The System Usability Scale contains 5-point Liket (strongly disagree to strongly agree) scale with ten statements which the respondents are required to respond to. The SUS for this study (appendix) is designed as a heuristic approach to usability testing measure, mainly for validation of the responses earlier provided. 3.11 Ethical Considerations Prior to starting the study, the researcher will seek to obtain an ethical approval from the University of East Anglia ethics committee. All study participants will be made aware about the study’s purpose, the activities they will undertake, the anonymity and confidentiality of the data, and their rights via a consent from. The consent will also include information on how the information will be used and will seek participants’ approval to publish the results of the study 3.12 Risks and mitigations measures The study will be conducted in a simple room set up and the materials used do not pose any hazard to the participants. Therefore, few if any safety measures will be considered or any risk will be estimated in advance and its mitigation measure established. Confidentiality of the information of the study will be assured to the participants as outlined in the ethics and considerations section. IV. SUMMARY Background: The ease of use of a product by a consumer can be quantified using usability tests. Usability requires a thorough understanding of the expected users of that product, their need, and the circumstances or environment of the product usage. The as human centred design approach of efficiency, effectiveness and satisfaction comprises the overall design process of a usability test. The choice of participants is widely believed to influence the results of the usability test. For this study, it is postulated that there is high likelihood of the participant’s personality to influence the overall usability testing experience. Precisely, the personality dimensions of extraversion-introversion have been claimed to have effect on user’s interaction with a computing product. Therefore, it is worthwhile for this study to establish the influence of the personality dimension on usability testing experience. Methods: For this study, an interactive product; University website will be used as the test object. The usability problems will be identified in the testing sessions as well as through heuristic evaluation. A desirable random sample of participants will be drawn from the population of users (students and staff) of the selected university website. Ten participants will be selected for the user testing exercise initially through the online Myers-Briggs Type Indicator. Five participants with high degree of extraversion (extraverts) and five participants with low degree of extraversion (introverts) will be chosen. The participants will be provided with overview of the ‘think aloud’ technique, usability evaluation method. Thereafter, the participants will be required to complete a set of four tasks used to identify usability problems. Ethical considerations will be observed for this study as it involves human participants. Expected results: It is expected that the study will establish whether the proposition that extravert provide better quantity as, well as quality assessment of usability compared to introverts. The study is expected to shed light into the disparities between the introverts and extraverts in the identification of usability problems. TABLE OF CONTENTS I.Introduction 2 1.1Background 2 1.2Statement of the problem 2 1.3The general objective of the research 3 1.4 Research questions 3 1.4Overview of research methodology 4 1.6 Challenges of research 5 1.7 Structure of research 6 II. LITERATURE REVIEW 7 2.1 The Definition of Usability 7 2.2 Usability Evaluation Methods (UEMs) 7 2.3 Factors effecting on usability testing 9 III interactive. RESEARCH METHODOLOGY 10 3.2 research method 10 This research is based on the experimental approach as this approach is known to be very effective in terms of achieving the research aim and objectives and answering its questions mentioned previously. 10 This research project will also involve quantitative and qualitative measurements. The research questions require the use of between-subject design in this study. This design is very useful in preventing learning effect between compared conditions. 10 3.3 Test object 10 3.4 Task 10 3.5 Participants 10 3.7 Data analysis 11 3.11 Ethical Considerations 11 3.12 Risks and mitigations measures 11 IV. SUMMARY 11 TABLE OF CONTENTS 13 APPENDIX 14 Work plan 14 REFERENCE LIST 15 APPENDIX Work plan Gantt chart Deliverables Number of Week April May June July 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 Finalising with the development of Research Proposal 2 Chapter One Improvements 2 In depth Literature Review 4 Preparation of Data collection tools and pilot study 3 Data Collection 3 Data Analysis 3 Results and discussion 3 Review of the dissertation and final submission 2 REFERENCE LIST Barendregt, W., Bekker, M. M., Bouwhuis, D. G., and Baauw, E., 2007. Predicting effectiveness of children participants in user testing based on personality characteristics. Behaviour & Information Technology, 26(2), pp.133-147. Barnum, C. M., 2010.Usability testing essentials: ready, set... test!.Elsevier. Bastien, J. C., 2010. Usability testing: a review of some methodological and technical aspects of the method. International Journal of Medical Informatics, 79(4), e18-e23. Bergstrom, J. C. R., Olmsted-Hawala, E. L., Chen, J. M., and Murphy, E. D., 2011.Conducting iterative usability testing on a web site: challenges and benefits. Journal of Usability Studies, 7(1), pp.9-30. Bevan, N., 2009. Usability. In Encyclopedia of Database Systems (pp. 3247-3251).Springer US. Britto, M. T., Jimison, H. B., Munafo, J. K., Wissman, J., Rogers, M. L., and Hersh, W., 2009. Usability testing finds problems for novice users of pediatric portals. Journal of the American Medical Informatics Association, 16(5), pp.660-669. Brophy, P., and Craven, J., 2007. Web accessibility.Library Trends, 55(4), pp.950-972. Burnett, G. E., and Ditsikas, D., 2006. Personality as a criterion for selecting usability testing participants. In Proc. Int. Conf. on Information and Communications Technologies (pp. 599-604). Capretz, L. F., 2003. Personality types in software engineering. International Journal of Human-Computer Studies, 58(2), 207-214. Dumas, J. and Redish, J. 1993.A practical guide to usability testing. Norwood, NJ: Ablex Publishing Company. Dumas, J. S., and Loring, B. A., 2008.Moderating usability tests: Principles and practices for interacting. Morgan Kaufmann. Federoff, M. A., 2002. Heuristics and usability guidelines for the creation and evaluation of fun in video games (Doctoral dissertation, Indiana University). Hartson, H.R., Andre, T.S., and Williges, R.C., 2001. Criteria for Evaluating Usability Evaluation Methods. International Journal of Human-Computer Interaction, 13(4), pp.373-410. Hertzum, M., Hansen, K. D., and Andersen, H. H., 2009. Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload?.Behaviour & Information Technology, 28(2), pp.165-181. Kortum, P. T., and Bangor, A., 2013. Usability ratings for everyday products measured with the System Usability Scale. International Journal of Human-Computer Interaction, 29(2), pp.67-76. Lindgaard, G., and Chattratichart, J., 2007. Usability testing: What have we overlooked?. In Proceedings of the SIGCHI Conference on Human factors in Computing Systems. ACM Press. Lindgaard, G., and Chattratichart, J., 2008. Usability testing: What have we overlooked?” in CHI 2007 Proceedings. ACMPress. Ludin, Z., 2011. On ethical problem solving in user-centred research: an analysis. In CHI11 Extended Abstracts on Human Factors in Computing Systems (pp. 791-798). ACM. Lund, A. M., 1998. The Need Fora Standardized Set of Usability Metrics. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 42, No. 10, pp. 688-690).SAGE Publications. Madan, A., and Dubey, S. K., 2012. Usability evaluation methods: a literature review. International Journal of Engineering Science and Technology, 4(2), pp.590-599. Molich, R., Thomsen, A. D., Karyukina, B., Schmidt, L., Ede, M., van Oel, W., and Arcuri, M., 1999. Comparative evaluation of usability tests. In CHI99 extended abstracts on Human factors in computing systems (pp. 83-84). ACM. Myers, I. B., McCaulley, M. H., and Most, R., 1985.Manual, a guide to the development and use of the Myers-Briggs type indicator.Consulting Psychologists Press. Nielsen, J., 1994. Guerrilla HCI: Using discount usability engineering to penetrate the intimidation barrier. Cost-justifying usability, pp.245-272. Peute, L. W., de Keizer, N. F., and Jaspers, M. W., 2013. Effectiveness of Retrospective and Concurrent Think Aloud in Formative Usability Testing; Which Method Performs Best?.Human factors methods in health information systems’ design and, 65. Peute, L. W., de Keizer, N. F., and Jaspers, M. W., 2015. The value of Retrospective and Concurrent Think Aloud in formative usability testing of a physician data query tool. Journal of biomedical informatics, 55, pp.1-10. Pickard, A., 2012. Research methods in information. Facet publishing. Quesenbery, W., 2003. The five dimensions of usability. Content and complexity: Information design in technical communication, pp.81-102. Redish, J., 2007. Expanding usability testing to evaluate complex systems. Journal of Usability Studies, 2(3), pp.102-111. Rosson, M. B., and Carroll, J. M., 2001.Usability engineering: scenario-based development of human-computer interaction. Elsevier. Rutherfoord, R. H., 2001. Using personality inventories to help form teams for software engineering class projects. ACM SIGCSE Bulletin, 33(3), pp.73-76. Similarminds, 2015.Personality Test (online). Accessed at: Accessed on [6 April 2015] Wixon, 2003. Evaluating usability methods: why the current literature fails the practitioner. interactions, 10(4), pp.28-34. Wixon, D., 2003. Evaluating usability methods: why the current literature fails the practitioner. interactions, 10(4), 28-34. Read More