Positivism vs Interpretivism, Induction versus Deduction - Research Proposal Example

Chapter 3: Research Methodology 3 Introduction This chapter focuses on the research methodology of the chosen research study. Several existing research approaches shall be discussed, along with comparisons and relations amongst them. Data collection techniques shall be discussed that facilitate the accumulation of data, alongside the pros and cons of each technique. After the evaluation and discussion of the research approaches, few of the research approaches shall be chosen and discussed in relation to the respective research study. 3.2 General Research Approaches The world of computing is encompassed of approaches and paradigms that facilitate the research processes and methodologies. The presence of a paradigm is relevant due to the following reasons: It serves as a guideline for the individuals who might be facing issues in a specific domain. It facilitates the development of models and theories that can prove to tackle the issues in an ordered manner. It facilitates the development of criteria that can prove to handle the prevailing issues. The development of processes and procedures can be achieved that might prove to manage issues in the future in a more organized manner. Some of the common paradigms that have been used in several research studies are discussed below: 3.2.1 Positivism vs. Interpretivism Weber described the differences between positivism and interpretivism on the basis of a concept known as ontology [96]. He explained ontology as the aspect that can explain the existence of anything; the researchers can study or research something if they have knowledge of the nature of the reality of the thing under discussion. Weber explained the difference between positivism and interpretivism; positivism dictates the assumption that the researcher and reality are separate, whereas interpretivism proposes that the researcher and reality cannot be separated [112]. Positivism is considered to be based on scientific reality and any concept within it can be comprehended from an objective point of reference. Whereas, interpretivism relies on the subjectivity of the concept to make the individuals comprehend it. The second aspect that facilitates the differentiation between positivism and interpretivism is epistemology. Epistemology refers to the source of knowledge and the theories of knowledge that can provide some relevant information. Positivism is based on the belief that objective reality is present beyond the intellect and thoughts of an individual, while interpretivism designates the source of knowledge as the real life experiences and social scenarios of the world [96]. The last aspect that forms the basis for comparison is the research object; it can be defined as the thing on which the study is being conducted. Positivism advocates that every research object has inherent values in them that exist independent of the research study, whereas interpretivism believes that every research object is perceived in the light of the personal experiences of the researcher [112]. It is due to these differences that different research methods are used to attain data regarding the objects that are under study: Positivism includes practical experiments at laboratories, field studies and surveys [112]. Interpretivism promotes case studies, ethnographic studies and phenomenographic studies as the ones that bear most effective results. 3.2.2 Induction versus Deduction Mackay and Fayard stated that deductive models aim to deduce the characteristic of aspects and objects found in real life from theories, whereas inductive model attempts to generalize the concepts and perceptions from real life speculations [97]. The following deductive model explains the processes that are involved in the respective model: Figure: Processes involved in a deductive model [97] The process of a deductive model begins with the development of a generalized phenomenon that might commonly exist in the real world. A hypothesis is generated that can be tested for validity with the aid of an experiment. It might be required to revise the hypothesis after conducting the experimentations. Figure: Processes involved in an inductive model The inductive process begins by making note of a phenomenon that might exist in the real world. A framework is developed to describe the phenomenon without the usage of any preconceived ideas. New observations can be made on the basis of questions that are generated on the previously recorded observations; the framework and model should be revised on the basis of new observations. 3.2.3 Quantitative versus Qualitative paradigms The quantitative paradigm takes support from positivism and bases the concept on the ontological point of view. It promotes that there exists a single truth that is also understood as objective reality; this truth is independent of the possible perceptions of humans. The researcher and the object (under study) are considered to be independent of each other such that they cannot influence each other at any level [113]. On the contrary, the qualitative paradigm is considered to take influence from interpretivism, however bases the concept on ontological point of view. It advocates that there exist multiple realities that might initiate as a result of the perception of reality by the individual. Another difference that becomes apparent between these two types of paradigms is that the researcher and the research object are interlinked, thereby facilitating the creation of mutual observations [113]. Figure: Three basic research paradigms, alongside mixed methods [95] As shown in the above figure, the extreme sides and the middle portion consists of pure research approaches, however the area that falls in the middle of these points tends to accumulate the mixed research approaches. The ‘Qualitative Dominant’ research approach focuses on qualitative research process; however the quantitative research processes are also recognized and believe that quantitative data can prove to benefit the research study. Similarly, the ‘Quantitaive Dominant’ approach tends to focus on the quantitative approach; however, believe in the inclusion of qualitative research processes since its inclusion might facilitate the attainment of even more effective findings. Mixed research processes tend to cater an accumulation of both the processes at varying degrees. 3.2.4 Exploratory versus Confirmatory Diana stated the differences between the two types of researches and stated that exploratory research process is adopted by the researcher when all the possible relationships are explained and explored with the aim of developing few relevant relations [114]. In this process, the relations are explored between the variables to devise the most suitable ones for the research study. On the other hand, the confirmatory research process aims to confirm the relations that have been devised between the variables in a model. 3.2.5 Summary of the Research Approaches After studying the different research approaches, different relations can be identified amongst them since there are multiple aspects where interrelations are witnessed. Positivism paradigm takes influence from deductive research processes, where most of the research processes are considered to fall under the quantitative paradigm. The relations in these paradigms are not confirmed as they are still being explored by the researcher, therefore they can be considered to fall under the domain of exploratory studies. On the same lines, interpretivism paradigm tends to take influence from induction research processes. Induction processes are known to amalgamate qualitative processes as well. Most of the studies that are conducted under these processes are considered as confirmatory studies. Quantitative/ Qualitative Inductive Deductive Positivism No relation Deductive research process is based on the generalization of a concept to a specific one [97]. Positivism aims to develop explanations and generalizations regarding theories in the world [99]. Quantitative paradigm inclines more towards positivism due to the belief of the existence of only one objective reality [113]. Exploratory research processes can be deductive in nature if “it attempts to ‘explain’, subject to testing” [116]. Interprevistism Inductive research processes are inclined towards confirmatory studies [115]. Interpretisivm also bases the theories on perceptions of the researcher therefore they may be multiple in nature [96]. Qualitative paradigm inclines more towards interpretivism since it highlights the possibility of multiple realities [113]. Confirmatory research processes are known to be inclined towards inductive processes since the research process begins with the formulation of alternate hypotheses [116]. No relation Figure : Relation of different research processes 3.3 Data Collections Techniques for general approaches Data is an important part of any research. There exist different methods through which data can be collected; each mode has its pros and cons. The following figure shows the data collections techniques that are usually witnessed in the respective types of research approaches, alongside their advantages and disadvantages: Techniques Description Advantages Disadvantages Interviews An interview can be defined as a detailed conversation with an individual or a group that attains useful and detail information and data [117]. Provides detailed information regarding the object or matter being studied [117]. Gives valuable information through face to face meeting [117]. In-depth interview of the individual on a personal level can help the interviewer to comprehend the situation at hand [117]. This technique might prove to be a time consuming process, specifically in terms of an in-depth personal interview [117]. Phone: The subject may be disinterested due to the occurrence of an atypical situation that the interviewer might be unaware of [117]. Email: Emails are not considered effective modes to conduct an interview since the individual might lose interest in the process if it consists of extensive questions [117]. Observations It is a method by which relevant information can be gained through observing behaviors and gestures of individuals [118]. Facilitates the provision of first hand information [118]. Gestures and behaviours of the subject help the interviewer understand the aspects that might not be communicated verbally. Time consuming Experts need to be hired to observe the behaviour of the subjects [118]. It cannot be done via email and phone. Survey Survey involves the feedback from participants regarding few pre-generated questions [119]. Surveys are usually conducted from a significantly greater number of participants. Greater extent of opinions and responses can be extracted in a shorter time [119]. A notion can be established on the basis of the results of the questionnaire since the sample may include a significant representation of a society. Email is an effective mode of conducting the survey since the subject can have the convenience of choosing his suitable time to fill it out. Email: Response rate may not be satisfactory [119]. Phone: The participants avoid instances of answering questions on the phone if the number of questions is greater [119]. However, phone will be a good choice to get responses for a short questionnaire. In-depth meeting: Most effective responses can be received from this mode, however it will prove to be a long and tedious process [119]. Focus Group This technique can be considered as an amalgamation of interviewing the participants, who possess similar attributes and interests [120]. This mode is considered to be easy and fast since the opinions of participants can be attained effectively through group discussions and analysis of their behaviours [120]. Attains responses and perceptions that might be difficult to achieve through any other mode of data collection [120]. In depth meeting will be beneficial for this type of collection of data. The aspect of biasness may influence the result of any research study [120]. It is often witnessed that group discussions are usually dominated by few individuals, thereby hiding the voice of others [120]. Phone and email cannot be used for this mode of collection. Document Studies These are the studies that are pre-written and have been conducted in the past [121]. This information will be available on a local basis and might be found in the local language of the region. It is an inexpensive mode of data collection. Chronological data can highlight significant trends in the changing times over the years. In depth meeting with a researcher or participants can be useful to verify the facts and findings of the document studies. Email can be used to transfer these document studies from one end to the other. The authenticity of the data may be questioned since the conditions under which the research took place are not known and the level of biasness of the researcher is not evaluated. Document studies cannot be studied or conducted via the phone. Table : Data Collection Techniques for General Purposes 3.4 HCI Research Approaches and Theories This research study is directed towards the evaluation of the experience of the user and the user himself, thereby making it related to the domain of human computer interaction. Therefore, this part of the research shall focus on different HCI research approaches and theories regarding the research methods such as: User-centered Approach/ Design It is very common for researchers to use a user-centered design for their models since it addresses the following aspects: Visual Design Interface design Information design Interaction design Functional Specifications User Needs Site Objectives [102] Abras, Krichmar and Preece stated that another important aspect that is found in user centered design is the usability testing that ensures the presence of the things that might be expected in a technology by the user [103]. Different Levels and Planes in the domain of Usability Aichholzer and Straub categorized the aspects into two main criteria; information oriented and task oriented [8]. The divisions of attributes with respect to the 5 planes proposed by Garrett are stated below [102]: Visual Design Interface design/Navigation Design/ Information design Interaction design/Information Architecture Functional Specifications/Content Requirements User needs/ Site Objectives Framework of Human Needs for User Experience Schulze and Krömker proposed a framework that accumulates some important factors regarding human needs to enhance the user experience: Relatedness Influence and Popularity Stimulation Competencies Security Autonomy It can be seen from these theories that the focus in the development of a technology is not solely kept on the scientific implementation, rather the design features and human requirements to improve the whole experience of the user. Mackay and Fayard considered HCI as a perfect amalgamation of both scientific implementation and design features of any technology or system, therefore HCI has been chosen as most suitable for this research study [97]. They also stated that HCI cannot be termed as a pure science since it also caters to the human aspects related to technology, rather than only naturally occurring events (that are usually witnessed in scientific practices). Figure: Inclusion of guidelines in the designing process of any system or technology [97] It can be seen from the above figure that the design research for any system involves the inclusion of guidelines that guide the researchers to improve the design in the light of the users’ requirements. This approach is commonly adopted by researchers in the current times. The following applications are among few of the examples: Hoven, Frens, Aliakseyeu, Martens, Overbeeke and Peters discussed the application of an innovative form of system that facilitated the process of recollecting things [105]. A scientific research took place that highlighted the way in which the mind works and memory is stored in the human mind. The scientific research findings concluded that human mind needs to be given cues in order to recall information and data. Due to which the presentation of cues became a part of the design research. Different options of cues were studied that would help people to recall data and varying approaches were weighed to present the cues in the most suitable manner. Therefore, the system was developed after the accumulation of scientific research and eventually, design research took place. Figure: Display of cues on the side to facilitate the viewing of the information [105] Hoven, Frens, Aliakseyeu, Martens, Overbeeke and Peters discussed another example of a technology that improved the product for the users due to the inclusion of scientific and design research [105]. The technology was aiming to provide service to the youngsters who were diagnosed with degenerative muscle disease. As a result of this disease, the patients were required to exercise the same workouts which became very boring after a certain time period. As an endeavour to make the technology interesting, design research was conducted to make changes in the technology to make it appealing for the young patients. As a result of the design research, video game activities were included in the workouts and were presented to the patients while they continued their exercises. The following figure includes the amalgamation of design and scientific research model that is known to produce effective results related to HCI: Figure: Amalgamation of design and scientific research model [97] The boxes in the above figure represent figures that are commonly found in the real world. Model: A research may devise a phenomenon from the realities of the world and then aim to develop a framework or model to explain it. Mackay and Fayard explained two models; deductive model and inductive model with respect to HCI practices [97]. The deductive model is defined as the most general model and it tends to explain the general theories and real world phenomenon. Inductive model observes the prevailing concept in the real world without being affected by the pre-defined theories and notions; it is after the materialization of the observations that a model is proposed to describe the phenomenon under discussion. Observations: Observations can be defined as the notions that are established as a result of examinations and analysis. Mackay and Fayard stated that scientists and researchers make use of observations and past theories to explain and present their newly developed theories to the world [97]. Villiers provided some information regarding the techniques that are used to evaluate online platforms; user’s thinking aloud, protocol analysis (verbal reports considered as important data) and walkthroughs [100]. Simulations: Yusoff defined simulations as the form of testing that enables the users to provide feedbacks on the basis of which changes can be incorporated in the system [16]. It is due to simulations that the mode of operations can be catered to the user’s requirements in an effective manner. Simulations may range from simple computer based experiments (for testing basic academic concepts) to technically advance virtual realities. Simulation Technique Description Advantages Disadvantages Computer games, Animations, high fidelity simulations [16] Computer games and animations can be used to facilitate the learning process and to educate the user about effective course of action in real life scenarios (which would be hard to replicate in real lives) [16]. These environments offer engaging scenarios and platforms [16]. Such technologies allow the user to be more productive [16]. Sometimes, this mode does not produce successful results [16]. Online conferences, chat messages [16] Learning experience can also be facilitated by online communication platforms like Skype, chat messages, etc [16]. Can be pursued at any time [16]. It can be done on a single user level or in a group [16]. Authenticity and relevance of such posts and comments needs to be analyzed. Simple computer simulations [123] Simple computer simulations can be used to replicate any scenario or functions of systems to attain the feedback of the users. This type of simulation allows the user to input some data in the system and analyze its results and workflows [123]. Learners get a free trial version to comment upon and use [16]. Simulation can be made to adopt different scenarios; however developers usually do not opt for such flexible modes of operability [125]. Online simulations [124] Online simulations are developed by individuals or companies that are made available for users and are based on a diverse range of academic, as well as professional concepts. It might be in the form of an interactive animation or application [124]. Students and professionals can understand concepts that might be difficult to grasp with the help of conventional modes [126]. Online simulations may inhibit the natural process of learning that is achieved in a classroom setting [126]. Table: Types of Simulations Prototypes: Prototypes are defined as the means of expressing the design artifacts of any system or technology before the whole product is released; this facilitates different design options in which the product can be developed [98]. There are different categories of prototypes. Few of the most common ones are role, integration, implementation and look and feel prototypes [122]: Role refers to the relevance of the technology or service in the lives of the users or the part that is played by the technology to facilitate any task in their lives. ‘Look and feel’ refers to the investigation regarding the design, experience that is attained after using the system or the technology. Implementation brings out the questions that are concerned with the techniques and modes adopted by the technology or system to perform its task; it mainly involves the strategies that are adopted by the technology to provide service to its users. Evaluation: Evaluation refers to the concept of analyzing the credibility and relevance of a system or technology for its users [101]. Evaluation assesses the benefits, limitations and constraints of the technology [101]. There are many types of evaluations; however the most common ones are user experience, user evaluation and expert evaluation. Users are the individuals who use the technology and face the possible limitations that might become apparent with due course of usage; the constraints might also be faced due to the uniqueness of their own traits and physical attributes. User evaluation tends to judge the relevance and suitability of the technology with respect to these personal aspects. There exist experts in every domain who have gained commendable knowledge in a certain field by means of experience of multiple years or research activities. Experts tend to evaluate technologies on the basis of their knowledge and their perceptions of good quality software. The expert might evaluate a certain technology in his own domain, for example a researcher who possesses extensive knowledge in the field of usability will be able to conclude relevant findings regarding the level of usability of the technology, rather than its security. Anderson explained a model that has been proposed by Liz Sanders; the prominent demarcations in this model are expert mindset and participating mindset, whereas the other prominent demarcations are design led and research led [109]. Figure: Design research model proposed by Sanders [109] The techniques proposed for each demarcation can be seen in the above figure and the degree to which overlap may exist between the respective demarcations. Psomas proposed the five competencies of user experience design that can facilitate the development of an effective design for any technology or system [111]: 1. Information architecture 2. Interaction design 3. Usability engineering 4. Visual design 5. Prototype engineering 3.4.1 UX Data Collections Techniques for HCI approaches User experience is one of the most common attributes for evaluating a certain technology or system. It is vital to evaluate the level of comfort, satisfaction etc that is being attained from a certain technology; such attributes and many others can be evaluated with the help of data collection techniques for user experience. The following table provides techniques that are suitable to measure the degree of user experience attained from any technology: UX Technique Description Advantages Disadvantages Eye tracking Eye tracking is a method of tracking the eye movement of an individual to evaluate how the user is shifting his point of focus from one point to the other and identifying the points where the user is looking at any point of time [106]. Eye tracking proves to be useful in evaluating how the users tend to relate and work with the designs of the site or the system [107]. The data needs to be made more reliable and accurate by means of filtering and analysis software [106]. A/B testing Attitudinal and Behavioural testing: This method describes the comparison between ‘what people say’ and ‘what people do’ [107]. This mode of data collection highlights the probable changes in the beliefs of a certain segment [107]. This mode of data collection proves to be useful solely to improve the design of a site [107]. Usability Benchmarking It can be explained as the study of a product on the basis of a predetermined plan to identify the usage aspects for a certain part of a system [107]. The specificity of the benchmarking study tends to conclude very useful and reliable findings [107]. The development of benchmarks may be different for different technologies and systems, therefore the development of the benchmark may prove to be a tedious task [107]. Message board Mining Online forums and message boards are considered to be important platforms for attaining the views of the individuals [108]. Top down methodology and bottom up methodology provide effective results for analyzing the message boards [108]. Manual analysis of the messages is not a preferable mode of evaluating the message boards [108]. The following figure includes few of these above stated data collection techniques to evaluate the level of user experience that is acquired after the usage of any technology or application: Figure : Data collection techniques to evaluate the user experience with any technology [107] Some user experience researchers have been wondering about the possible theories behind the UX methods and approaches. As a result of their analysis and investigation, the following have been concluded in the SIG session [110]: 1. Lab studies with individuals 2. Lab studies with groups 3. Field studies (short, e.g. observation) 4. Field studies (longitudinal) 5. Surveys (e.g. online) 6. Expert evaluations 7. Mixed methods 3.4.7 Chosen Approaches for the Research Study It is important for researchers to identify the differences between approaches while aiming to choose the ones that provide a chance to integrate different methods suitable for a certain research study. The following table includes the summary of the approaches that have been chosen to answer the research questions for the proposed research study: Main Tasks General Research Approaches Specific Data Collection Techniques HCI Approaches and Theories UX Data Collections Techniques for HCI approaches Research question 1 Observation of the current issues and needs Interpretivism Paradigm= Induction Research = Most of these are Qualitative Paradigms = Confirmatory Studies Document Studies In these research questions, only scientific research is being performed. It starts from observations and goes onto design framework. Then, evaluation of the framework takes place using expert evaluations. it is considered as an induction research process as shown in the figure above. It is considered to be a confirmatory study because it seeks to confirm a pre-specified relationship. In the light of these arguments, it can be stated that these two questions do not need any selection of any HCI approaches or UX data collection technique since it is a clear scientific research process without the involved of any human interaction. Research question 2 Designing and evaluating a framework Focused group of experts from the three perspectives. These experts’ evaluations are used via online questionnaires and video tutorials Research question 3 Designing and evaluating a model Positivism Paradigm = Deductive Research = Most of these are Quantitative Paradigms= Exploratory Studies There will be mixed methods. Designing and evaluation of TAM Model using user evaluation that will involve questionnaire and interviews after performing some tasks according to certain scenarios. To answer these two questions, HCI processes and approaches shall be used that involve scientific, as well as designing research. Task analysis Research question 4 Designing, evaluating a system and comparing it with other existing systems. Using user evaluations, it is planned to conduct a survey through questionnaires and interviews after performing some tasks in the computer prototype of the system. Designing, evaluating a computer prototype using a lot of UX data collection methods (that have also been mentioned in the previous section such as A/B testing and task analysis) Table: Chosen Research Approach Figure: Inductive model and the processes involved in it [97] With reference to the above figure, first and second question shall be researched in the phases of observations and framework. Observations phase shall be discussed in detail in Chapter 2 that will also discuss the current issues related to the research study. Chapter 4 will contain the relevant theories and suitable attributes for the development of the framework. Chapter 5 shall contain comparisons between existing frameworks that are related to the domain under discussion; however, details regarding the development of the framework shall be discussed in Chapter 6. Chapter 7 shall contain the evaluations and validations related to the researched frameworks. Figure: Movement between theory and observations in HCI practices [97] With reference to the above figure, research question 3 and 4 shall be addressed in the model and system processes. Chapter 8 and 9 shall include the details and information regarding the selection, extension, designing and evaluation of the model. Chapter 10 shall include designing and evaluation of the proposed system with respect to the existing systems. References [95] K. I. Al-Qeisi, “Analyzing the Use of UTAUT Model in Explaining an Online Behaviour: Internet Banking Adoption”, Brunel University, 2009. [96] R. Weber, “The Rhetoric of Positivism Versus Interpretivism: A Personal View”, MIS Quarterly, Vol. 28, No. 1, 2004. [97] W. E. Mackay and A. L. Fayard, “HCI, Natural Science and Design: A Framework for Triangulation Across Disciplines”, Designing Interactive Systems, 1997, pp. 223-234. [98] S. Houde and C. Hill, “What Do Prototypes Prototype?”, Handbook of Human-Computer Interaction, 2nd Ed. [99] A. Mewse, “An Introduction to Qualitative Research Methods”, University of Exeter. [100] C. Villiers, “Using HCI techniques to evaluate Electronic Commerce sites”, University of Pretoria, 2000. [101] L. Sharp and J. Frechtling, “User-Friendly Handbook for Mixed Method Evaluations”, Division of Research, Evaluation and Communication, 1997. [102] J. J. Garrett, "The Elements of User Experience: User-Centered Design for the Web and Beyond, Second Edition", 2000. [103] C. Abras, D. M. Krichmar and J. Preece, “User-Centered Design”, In Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications, 2004. [104] K. Schulze and H. Krömker, “A Framework to Measure User Experience of Interactive Online Products”, Proceedings of Measuring Behavior, 2010. [105] E. Hoven, J. Frens, D. Aliakseyeu, J. B. Martens, K. Overbeeke and P. Peters, “Design Research & Tangible Interaction”, Proceedings of the First International Conference on Tangible and Embedded Interaction, 2007. [106] A. Poole and L. J. Ball, “Eye Tracking in Human-Computer Interaction and Usability Research: Current Status and Future Prospects”, Encyclopedia of Human-Computer Interaction. Pennsylvania: Idea Group, 2005. [107] C. Rohrer, “When to Use Which User Experience Research Methods”, Jakob Nielsens Alertbox , 2008. [108] N. Glance, M. Hurst, K. Nigam, M. Siegler, R. Stockton and T. Tomokiyo, “Analyzing Online Discussion for Marketing Intelligence”, World Wide Web Conference, 2005. [109] R. Anderson, “User (Experience) Research, Design Research, Usability Research, Market Research: A Changing, Interconnected World”, UX Magazine, 2011. [110] V. Roto, All About UX, n.d. [111] S. Psomas, “The Five Competencies of User Experience Design”, UX Matters, 2005. [112] R. Weber, “The Rhetoric of Positivism versus Interpretivism: A Personal View”. MIS Quarterly, Vol. 28, No. 1, 2004. [113] J. E. M. Sale, L. H. Lohohfeld, K. Brazil, “Revisiting the Quantitative-Qualitative Debate: Implications for Mixed-Methods Research”, Quality & Quantity, Vol. 36, 2002, pp. 43–53. [114] D. S. Diana, “Exploratory or Confirmatory Factor Analysis?”, SUGI. 31 Proceedings, Paper No. 200-31, 2006. [115] R. S. Silvestre1, T. H. C. Pequeno, “Confirmation Theory and the Logic of Inductive Implication”, University of Montreal, 2005. [116] R. G. Jaeger, T. Halliday, “On Cnfirmatory versus Exploratroty research”, Herpetologica, Vol. 54, 1998, pp. S64-S66. [117] L. K. Owens, “Introduction to Survey Research Design”, SRL Spring 2005 Seminar Series, 2005. [118] T. C. Lethbridge, S. E. Sim, J. Singer, “Studying Software Engineers: Data Collection Techniques for Software Field Studies”, Empirical Software Engineering, Vol. 10, 2005, pp 311–341. [119] J. K. Doyle, “Face-to-Face Surveys”. Encyclopedia of Statistics in Behavioral Science, Worcester Polytechnic Institute, 2005. [120] Department of Health and Human Services, “Data Collection Methods for Program Evaluation: Focus Groups”, Evaluation Briefs, Vol. 13, 2008. [121] Y. D. Plessis, “Chapter 4: Research Methodology”, University of Pretoria, 2004. [122] S. Houde, C. Hill, “What Do Prototypes Prototype?”, in Handbook of Human-Computer Interaction (2nd Ed.), 1997. [123] R. Thomas, “What Are Simulations? – The JeLSIM Perspective”, JelSIM, 2003. [124] R. L. Bell, L. K. Smetana, “Using Computer Simulations to Enhance Science Teaching and Learning”, Technology in the Secondary Science Classroom, 2008. [125] R. Min, “Advantages and disadvantages of model-driven computer simulation”, University of Twente. [126] K. L. Smart, J. J. Cappel, “Students’ Perceptions of Online Learning: A Comparative Study”, Journal of Information Technology Education, Vol. 5, 2006. Read More