Why Should Software Practitioners Conduct Empirical Evaluations - Assignment Example

?Question Why should software practitioners conduct empirical evaluations? First and foremost, the term empirical evaluation refers to the scientific manner in which one can prove a certain theory by carrying out an experiment or through observation. So why should software practitioners carry it out? Empirical evaluations can help in the identification of errors. Empirical evaluations help to identify errors in a software an one is able to rectify such errors therefore making the software run more efficiently and effectiviely. Also, empirical evaluations contain mathematical proofs illsutrating that developed systems can function thus increasing confidence among users in the reliability of a system (Boehm 2008). The systems developed are inherently complex by nature not to mention that they are constantly changing and this has in a large part required empirical evaluations to be carried out especially with regards to the large systems so as to incorporate continuity. Question 2 What factors might make it difficult to conduct an empirical evaluation of the scenario? Explain five factors and relate each of them to the scenario. There are several factors that might make it difficult to conduct an empirical evaluation in this scenario. They include: i. The researcher’s inexperience. The complex nature of carrying out such tasks requires one to have experience in this field. In this scenario, the person concerned has not even studied this subject and this coul prove to be a mjor stumbling block. ii. Inadequate resources. In order to carry out an empirical evaluation, large amount of resources are required. In our scenario, it might not be possible to acquire adequate funding to carry out such a project and therefore transforming theory into reality via empirical evaluation might not be possible. iii. Cumbersome. The whole process of carrying out an empirical evaluation is very cumbersome as it requires one to put in a lot of effort and deidcate a lot of time to such a project. Without determination, the end product might never be seen and this could be a major hinderance in this scenario. iv. Finding the correct algorithms. So as to prove the correctness and effectiveness of a system, the developer will require proven algorithms. Finding the correct algorithm can prove to be a daunting task. v. Inadequate information.Another difficulty might come about when seeking further information with regards to the two methods. There is very little information on the matter as there has been little or no reaserch done cocerning this. Question 3 Using the Fenton and Pfleeger model, why is it hard to show that an lntegrated Development Environment (such as those stated in the scenario) leads to improvements in the software project, or in the quality of the software produced. The end product in software production is greatly dependent on two factors; the experience of the developers and the input applied to the development process. A high quality software at the end of the day should do what it was intended for but in a simpler manner and for this to come to fruittion, it depends largely on the design principles and the developer. The Integrated Development Environment lays more emphasis on the ease of use, manageability and user interfaces. This makes it more difficult to evaluate the importance of determining the quality of the controls, resources and input since they are determined by the developers. Further, in order to determine the output, provision of empirical evidence is key so as to predict it. Coming back to our scenario, this is not practical because of the complexities and numerous functionalities of the environments (Lewis & Veerapillai 2005) Question 4 Critically discuss the benefits and limitations of the case study and the survey study to answer'- the friend's question. There are many benefits that can be realised through conducting a case study method in the evaluation process. Some of these benefits may include; quantifying of the various benefits and functionalities that may be realised in a single project. This means that the developer will not need to constantly repeat the evaluation process and as a result will save valuable time and costs on the evaluation process. Another benefit realised will be that this method allows the developer to see the benefits and functionality of the project before its completion. As a result, the developer will be able to work within a certain deadline and thereby allowingthe developer to have a stable development procedure and schedule. This method is not perfect and it therefore has its limitations. For instance; when comparisons are made between a case study and the actual project we run a risk of limiting the process to only those benefits that were observed in the single case study carried out. Other benefits might not be realised. If not properly handled, the user might be presented with unrealistic impressions. It is also important to note that a case cannot be applied to every other projects, they might differ in some aspects and is therefore prudent to realise that application should not be blanket but dealt with on a case by case basis. The Benefits and Limitations of the Survey Method A survey is a data collection method where individuals who have employed a certain method/tool on various projects are asked to give their opinion on its benefits and functionality. For example, in the above scenario, in order for the student to gain a better understanding of which one to use between the BlueJ and NetBeans environments and which is the best method to apply. One of the main benefits of this method is that a wide range of information can be acquired and at the end one is better placed to make a decision. This method is very easy as it does not include very many steps. One is merely required to interview the identified experts in the field and pose to them the relevant questions. In this method, the reasercher is in total control as he identifies the people he wants to gain information from, sets the questions to ask and seek clarification incase of any ambiguity. A major disadvantage of using this method in the scenario is that individual surveys do not provide one with accurate data.For one to acquire good and relevant information, one has to carry out several surveys on the different methods/tools. Lastly, respondents may be biased and therefore the information provided by respondents may be skewed. Question 5 Define the terms "claim", "argument", "evidence" and "empirical evidence", and explain how they relate to each other. A claim is an arguable statement. It is an idea or statement that can either be true or false. This term can be used in various contexts to give different meanings. The person who makes the claim usually is of the belief that the claim is true and therefore tries to convince others of the same. The term evidence refers to a body of facts or information that is used to indicate whether a certain proposition or claim is true or valid. Therefore in order to prove whether a claim is true or not, evidence should be provided. Without the provision of evidence, then claims can be relegated to mere allegations. What is an argument? It can be defined as a coherent series of statements that leads from a premise all the way to a conclusion. In other words, it is a combination of various claims that are related and evidence that supports them. Empirical evidence can be explained as a provable fact whose results cannot be questioned. It is proof that knowledge or information was gained by data, and not through hypothesis, or conjecture.In tying all these terms together and showing their relatioship, upon making certain claims, one can only prove their validity by providing empirical evidence. The evidence helps make ones case case stronger as they make their arguments as to why their claims are true. In a nutshell, arguments rely on empirical evidence in oder to ascertain whether certain claims are true or false (CONFERENCE ON EVALUATION AND ASSESSMENT IN SOFTWARE ENGINEERING 2002). Question 6 Briefly explain the difference between Evidence Based Software Engineering (EBSE) and Systematic Literature Reviews (SLR). A systematic Literature Review is a method that mostly focuses on three major aspects of research; identifying research, evaluating research and lastly, giving accurate interpretations of research that is both available and relevant to the software project in question. Whereas EBSE on the other hand, can be decsribed as a method primarily used in software development and evaluation, which applies the various knowledge and evidence collected in order to ascertain whether projects are reliable and relevant. Its main aim is to collect evidence materials from various researches and use the information acquired to approve projects. Question 7 Critically discuss the limitations of EBSE as a methodology for evaluating lntegrated Development Environment. EBSE experiences quite a number of limitations as a methodology for evaluating Integrated Development Environment. One of the major causes of the difficulties experienced is due to the little or completely lack of research conducted in this area over the years. This has been a major hinderance as practioners in relevant fields lack a point of reference. Also, the information collected in this field is reported in a manner that makes it very difficult to accumulate and evaluate the kind of evidence offered. This makes it extremely difficult for practitioners especially novices in this field to fully comprehend the information and therefore applying it is even more difficult if not impossible. As one of its procedures, EBSE recommends that a literature review be carried out and the information collected ought to then be integrated with the developer’s experience. Intergrating the literature review with the developer’s experience can prove to be a stumbling block especially in this scenario where the developer is new in the field and lacks substantial practical experience for integrating the research. Lastly, carrying out literature reviews from the internet is very difficult. This is beacause the quantity of information can be overwhelming and at times it is very difficult to select the proper type of information. (International workshop on realising evidence-based software engineering 2000). REFERENCES BOEHM, B. W. (2008). Characteristics of software quality. Amsterdam, North-Holland Pub. Co. BHUTA, S. P. (2009). First encounter with Java Including BlueJ. Mumbai, India, Shroff. BUDGEN, D. (2000). Special issue on: evaluation and assessment in software engineering. Amsterdam, North-Holland. Conference on Evaluation and Assessment in Software Engineering. (2002). Special issue on empirical software engineering. London, IEE. LEWIS, W. E., & VEERAPILLAI, G. (2005). Software testing and continuous quality improvement. Boca Raton, Auerbach Publications. International Workshop on realising Evidence-Based Software Engineering. (2000). Kitchenham, B.A. Pickard, L.M. (2001). Case studies for method and tool evaluation. IEEE Software, pp52-62. NAIR, P. S. (2009). Java programming fundamentals: problem solving through object oriented analysis and design. Boca Raton, CRC Press. Read More