Importance of Testing in Software Engineering - Research Paper Example

12 November 2008 Software Testing I have chosen the Software Testing a topic for my assignment of literature review. In this literature review my aim is to elaborate software testing related topics in the detail. So for this reason I will analyze all the aspects of this topic to have better understandings of what Software Testing is. In the first section of this literature review I will explain what the Software Testing is in the project life cycle stages then I will provide a little history of the Software Testing. I will try to analyze its significance for the project prospective and for the project enhancement prospective. Then in the next section I will explain main point for the Software Testing that is error. It means we need Software Testing when there is a chance of error. So in that section I will explain different types of errors and elaborate how they can occur so that we can minimize the factors that are sources for the happening of those errors. In the next section I will present different testing techniques. After that I will explain Software Testing process. The proceeding section will contain the different testing tools. Challenges that are running with this field are discussed in the next section. Then I will conclude the whole discussion and at the end work cited will be provided. Testing is a necessary action in software engineering. In the easiest terms, it amounts to viewing the implementation of a software system to authenticate whether it performs as planned and recognize possible malfunctions (Bayse et al, 2005). Testing is extensively utilized in industry for superiority assertion: indeed, by straightly scrutinizing the software in carrying out, it offers a practical feedback of its performance and as such it remains the unavoidable complement to other analysis methods. Beyond the apparent straightforwardness of checking a model of runs, though, testing embraces a diversity of behaviors, techniques and performers, and poses several complex challenges. Definitely, with the difficulty, pervasiveness and criticality of software growing incessantly, guaranteeing that it acts according to the desired altitudes of excellence and dependability becomes extra critical, and increasingly tricky and expensive. Earlier studies anticipated that testing is able to consume 50 %, or still more, of the development costs, and a new detailed study in the United States enumerates the high economic impacts of an insufficient software testing infrastructure (Bayse et al, 2005). History: The partition of paradigm of debugging from testing was formerly instigated by Glenford J. Myers in year 1979. Even though his absorption was on breakage testing that is an unbeaten test is one that discovers a bug, it demonstrated the wish of the software engineering society to split fundamental development actions, like that debugging, from that of confirmation. So that seeing these urges Dr. Dave Gelperin and Dr. William C. Hetzel classified in 1988 the stages and objectives in software testing in the following phases: given below: (Yilmaz,, 2004). Until 1956 - Debugging oriented 1957-1978 - Demonstration oriented 1979-1982 - Destruction oriented 1983-1987 - Evaluation oriented 1988-2000 - Prevention oriented Why Testing? It is an important question that can be emerged in the mind of any person before going in further details. For answering this question I have extracted few points that demand incorporation testing criteria in project development for the complete assurance that it’s going well. The Open Source Security Testing Methodology Manual has stated several factors that necessitate the testing. The OSSTMM needs that risk incomes those restrictions in the safety presence will have a detrimental result on citizens, society information, procedure, commerce, image, rational property, lawful rights, or academic capital. The Open-Source Security Testing Methodology Manual maintains four elements for which we must have testing to minimize and cut down even the negligible risk state situation: (Beizer, 1990) 1. Safety We need security in our application. Like a bank application would not abide the security leakages. So we have to tackle this feature. All tests have to exercise apprehension for nastiest case situations at the maximum expenses. It necessitates the tester to grasp above all else to observe for human security in physical and affecting health and profession (Belinfante et al, 2001). 2. Privacy All tests have to work out regard for the exact individual privacy apart from the regional law. The ethics and sympathetic for isolation are frequently additional advanced than present legislation (Herzog, 2006). 3. Practicality We have to bring the Practicality in our developed product and for this we need so testing criteria to make it sure that it is working properly. All tests have to be engineered for the most negligible complexity, utmost viability, and genuine clarity. 4. Usability All tests have to stay inside the frame of working security. Majority safe is the smallest amount friendly and forgiving. The tests inside the Open-Source Security Testing Methodology Manual are carried out to look for a working level of safety. 5. Perfect Security In risk appraisal, the OSSTMM is appropriate to the method of Perfect safety. In ideal Security, the tester and analyst determine the customer as to what would be ideal security. This is countered with the bearing Review, which is most excellent, carry out, the client’s manufacturing system, the client’s commerce explanation, the client’s safety policy, and the lawful matters for the customer and the client’s areas for responsible business. The consequence is ideal Security for that customer. The tester and analyst then offer a space analysis among the present condition of safety with faultless Security (Herzog, 2006). After analyzing that why we need testing, we have to find out what are the different reasons that turn out the need of testing. So for this I have found a few types of errors that can disturb the working of the application are can effect the project development life cycle. I have mentioned how the auditor/or tester can identify these errors. One impotent thing to mention is these errors can happen due the human or computer tools or machinery malfunction. (Berner et al, 2005) Error Types1 False Positive The target rejoinder points out an exacting circumstance as factual though in authenticity the circumstance is not accurate (Leon et al, 2005). A fake optimistic often happen when the auditors viewpoint or suppositions of what points to a scrupulous state does not grasp to real-world circumstances which are hardly ever black and white (Herzog, 2006). False Negative The target rejoinder points to a scrupulous condition as not factual although in realism the state is factual. A fake negative normally comes about when the auditors outlook or suppositions regarding the target does not grasp to real-world state of affairs, the tackle are the inaccurate kind for the test, the tools are distorted, or the assessor be shorts of experience. A false negative preserve be hazardous as it is a misdiagnoses of a protected condition when it does not survive (Herzog, 2006). Gray Positive The target rejoinder indicates a challenging state as factual though the target is projected to act in response any reason with this condition whether it is or not. This variety of safety throughout anonymity may be unsafe as the delusion cannot be certain to effort the similar for all stimuli (Herzog, 2006) Gray Negative The target response points to a particular state as not accurate though the objective is intended to react to some basis with this state whether it is or not (Jonah, 2003). This class of safety throughout insignificance may be hazardous as the delusion cannot be definite to work the similar for all incentive. Specters The target reply indicates an exacting state as either true or false though in practicality the state cannot be prominent. A specter commonly happens when the auditors obtains an answer from an outer motivation that is apparent to be from the intention. A specter might be also deliberate of the goal, a wrongdoing from inside the channel, or the consequence of negligence and/or greenness from the auditor. One of the for the most part troubles in the echo procedure is the supposition that the reply is an outcome of the test. And result testing in the genuine world cannot accomplish time after time dependable consequences because neither the reason nor the outcome is able to be appropriately isolated. Indiscretion The target reply designates a complex condition as well honest or fake but only during a challenging time. That juncture may or may not go at the back a prototype and if cant be recognized at a time when the situation changes, it may reason the auditor to not understand the other state. An assessor may also decide that this is an irregularity or a difficulty with testing paraphernalia predominantly if the assessor unsuccessful to standardize the equipment previous to the test and carry out suitable logistics and controls. Injudiciousness is able to be unsafe as it might guide to a false exposure of the state of safety. Entropy Errors The target reaction cannot in fastidious point to an exacting state as moreover true or fake owing to a high racket to indication ratio (Hazelhurst et al, 2001). Similar to the thought of behind a flashlight grin in the sunlight, the examiner cannot correctly decide condition pending the noise is concentrated This kind of environmentally reasoned error hardly ever survives in the lab though is a usual incidence exterior of the lab in an unrestrained surroundings. Entropy can be unsafe if its belongings cannot be opposed. Falsifications The target rejoinder indicates a thorough situation as moreover factual or forged though in pragmatism the situation is needy leading primarily unidentified variables owing to goal bias. This kind of safety throughout darkness may be hazardous as the prejudice will move when tests approach from diverse vectors or employ dissimilar methods. It is as well probable that the objective is not conscious of the bias. Sampling Errors The target is a biased example of a bigger system or a better amount of likely states. These mistakes generally happen when an authority pressures the prepared condition of the objective for the interval of the test (Fyodor, 1997). This might be throughout specific time restraints on the test or a partiality of testing simply that which is designated as noteworthy inside a system. This kind of mistake will reason a parody of the in general operational safety. Constraint The boundary of human sanity or apparatus means points out a rigorous state as furthermore factual or false though the explicit state is unidentified. This error is not reasoned by deprived judgment or incorrect tools options rather it is a malfunction to be familiar with imposed restraint or limitations. Propagation The auditor does not form an exacting test nor has a preconception to pay no attention to a particular result owing to a supposed outcome. This is frequently a blinding from knowledge or a conformational bias. The test could be repeated many times or the tools and paraphernalia may be customized to have the preferred result. As the name means, a procedure which take deliveries of no criticism and the mistakes stay put unidentified or unnoticed will spread additional mistakes as the testing carry on. Propagation errors may be unsafe for the reason that the errors broadcast from near the beginning in difficult may not be noticeable throughout an analysis of termination. In addition, a study of the whole test procedure is necessary to find out spread errors. Human Errors The errors reasoned by are deficient in skill, knowledge, or understanding, is not one of bias and is forever a problem and forever nearby in spite of method or methods. Where a knowledgeable auditor may make proliferation faults, one devoid of knowledge is additional likely not to be familiar with human error, amazing which knowledge educates us to be recognizable with and recompense for. Statistically, there is a not direct association among knowledge and human error. The fewer experiences an auditor has, the better the quantity of human error a review will hold (Farmer et al, 1995). After analyzing different error types we have also few security limitations. And testing is the only solution to overcome these limitations. This section of the literature review will present the main security limitations that can affect the IT security like network, firewalls; other IT related projects. Why we study these limitations? Answer is: these limitations are the main source of error in any software development life cycle. So we can avoid them by knowing about them. Security Limitations: The condition of safety in looked upon to recognized faults and security limits inside the assortment are intended as Limitations (Kenneth, 1998). By setting proper principles to every restriction kind, they have to be classify and classified. While some classification name or numeral are able to utilized, this methodology tried to name them according to their belongings on OPSEC plus Controls and does not stare them in a hierarchal arrangement of sternness. Five categorizations are chosen to stand for all kinds of limits (Kenneth, 1998). Vulnerability: It is a fault or inaccuracy that refutes admission to possessions for official people or procedures, permits for honored admission to possessions to prohibited people or procedures, or permit prohibited public or procedures to cover assets or them inside the scope. Limitation:2 It is a fault or error that concerns, diminishes, abuses, or invalidates intentionally the belongings of the interactivity controls verification, indemnification, confrontation, overthrow, and permanence. Anxiety: It is a fault or error that disturbs, decreases, abuses, or invalidates the effects of the course or completion of practice controls non-denial, discretion, solitude, honesty, and alarm (Herzog, 2006). Experience: It is an indefensible work, fault, or inaccuracy that offers straight or roundabout visibility of plans or possessions within the preferred variety channel of the security attendance (Herzog, 2006). Anomaly: It is some unidentifiable or unidentified component which cannot be counted in typical processes. A constraint is a limitation if it acts in one of the precautionary issues as explained here. A good reason for a restraint is a risk outcome and one that is also convened with a control of a number of kinds yet if that control is only acceptance. Risk choices that suppose the confines as they are normally come downward to: the damage a control can not defend the price to fix or manage the restraint, the restraint have to be so according to legislation, system, or strategy, or a termination that the danger does not exist or is probable for the exacting inadequacy (Barisani et al, 2001). Software Testing Types3 This section of the literature review provides the Software Testing Types that are announced by the Open Source Security Testing Methodology Manual. Software Testing is an umbrella appearance to comprise all shapes and methods of safety experiments from the interruption to the practical audit. The function of the method from OSSTM manual will not dissuade from the selected kind of testing (Herzog, 2006). Though, as a normal, this methodology is not to be going after off-the-shelf. Realistic execution of the OSSTMM needs most imperative entity testing applications to set up the necessities given here. This income that still when pursuing this methodology, your application of it and your process will reproduce the kind of test you have selected Test kinds may be, but are not partial to, one of these 6 common kinds: (Herzog, 2006). Blind The auditor connects the target with no previous knowledge of its ramparts, possessions, or straits. The target is standing by for the review, noteworthy in proceed all the particulars of the audit. A sightless audit primarily tests the abilities of the auditor. The width and depth of a blind audit can merely be as huge as the auditors appropriate information and proficiency permits. In COMSEC and SPECSEC, this is frequently referred to as Ethical Hacking and in additional PHYSSEC and HUMSEC strait this is usually scripted as War Gaming or position playing. Double Blind The assessor connects the goal with no earlier knowledge of its resistance, possessions, or channels. The goal is not well-versed in advancement of the series of the audit, the controls tested, or the test vectors. A dual blind review tests the ability of the assessor and the attentiveness of the goal to unacknowledged variables of campaigning. The width and deepness of a blind audit be able to merely be as massive as the auditors appropriate knowledge and competence permits. This is as well recognized as a Black-Box-Audit or Penetration-Test. Gray Box The auditor employs the goal with partial knowledge of its defenses and possessions and filled acquaintance of channels. The mean is equipped for the review, consequential in proceed all the particulars of the review. A gray box review tests the abilities of the assessor and the attentiveness of the objective to unidentified variables of campaigning. The temperament of the test is competence. The width and deepness depends upon the superiority of the information supplied to the auditor previous to the test in addition to the auditors proper knowledge. This kind of test is frequently referred to as a Vulnerability Test and is the majority frequently begin by the target as a self appraisal (Herzog, 2006). Double Gray Box The auditor appoints the target with limited knowledge of its defenses and resources and absolute knowledge of channels. The goal is to inform in proceed of the range and time support of the audit but not the conduits tested or the experiment vectors. A dual gray box audit assessment the skills of the assessor and the targets attentiveness to unidentified variables of campaigning. The temperament of the test is competence. The width and deepness relies upon the excellence of the information offered to the auditor and the target previous to the test in addition to the auditors applicable information. This is as well recognized as a White Box Audit (Petrenko et al, 1994). Tandem The auditor and the purpose are standing by for the audit, together expressive in progress all the particulars of the audit. A tandem audit assessments the protection and manages of the target. It cannot test the attentiveness of the target to unidentified variables of campaigning. The factual nature of the experiment is concentration to detail as the assessor does include filled sight of all tests and their replies. The breadth and depth relies upon the superiority of the information given to the auditor previous to the test (clearness) as well as the auditors appropriate knowledge. This is frequently recognized as an In-House Audit or a Crystal-Box-Audit and the assessor is normally part of the safety process. Reversal The auditor employs the target with absolute knowledge of its observation and equipped security, although the target is well-known with nothing of what, how, or at what time the assessor will be testing. The right nature of this test is to audit the attentiveness of the target to unknown variables and vectors of demonstration. The width and deepness relies upon the superiority of the information offered to the auditor and the auditors appropriate knowledge and originality. This is as well frequently called a Red Team work out. In the occasion of treatment the audit, it is necessary to recognize precisely the kind of audit executed. Too often, reviews of diverse test categories are contrasted to follow the delta (divergence) from a recognized baseline of the span. If the accurate test kind is not obtainable to a third-party commentator or regulator, the audit itself should be measured a Blind test, which is one with the least advantage towards a methodical security analysis (Herzog, 2006). Black box testing Black box testing takes care of the software as a black box without any knowledge of interior implementation. Black box testing techniques comprise correspondence partitioning, limit value analysis, all couples testing, fuzz testing, replica based testing, and requirement based testing (Bernot, 1991). White box testing White box testing, by difference to black box testing, is at what time the tester has entrance to the inner data structures and algorithms and the code that application scenario (Marchetti at al, 2004). Kinds of white box testing code coverage Code entirety evaluation Mutation testing techniques Fault injection techniques static testing . Testing process Now I will explain the testing process applied for the removal and fixation of the different errors. A widespread operation of software testing is carried out by a self-governing group of testers subsequent to the functionality is urbanized earlier than it is transported to the client. This application frequently outcomes in the testing stage being utilized as project buffer to recompense for project stoppages, thus compromising the time dedicated to testing. One more application is to instigate software testing at the similar instant the project begins and it is an continual process in anticipation of the project ends.\ (Petrenko et al, 1994). In counterpoint, a number of capable software regulations like that tremendous programming and the approachable software development association hold on to a test-driven software development representation. In this practice unit tests are written primarily, by the software engineers frequently with couple programming in the tremendous programming method. Certainly these tests are unsuccessful originally; as they are predictable to. Then as code is developed overtakes incrementally better pieces of the test suites. The test groups are incessantly updated as novel failure conditions and bend cases are exposed, and they are incorporated with some regression tests that are urbanized. Unit tests are preserved next to with the rest of the software source code and in general included into the build procedure (with intrinsically interactive tests being transferred to a moderately physical build receipt process) (Hetzel, 1988). Testing can be complete on the following stages: (Roger, 2001) Unit testing tests the slight software part, or module. Each module (can say each basic part) of the software is tested to authenticate that the comprehensive design for the component has been properly implemented. In an object oriented atmosphere, this is typically at the class intensity, and the smallest unit tests comprise the constructors and destructors. Integration testing shows imperfections in the interfaces and communication among incorporated mechanism (these can be different modules). Increasingly bigger collections of tested software components equivalent to rudiments of the architectural design are included and tested in anticipation of the software mechanism as a system. System testing tests a totally incorporated system to confirm that it assembles its requirements. System integration testing confirms that a system is incorporated to some outside or third party systems distinct in the system requirements. Previous to deployment of the finishing version of software, alpha and beta testing are frequently completed as well: Alpha testing is replicated or real operational testing by possible consumers/customers or an self-governing test groups at the developers location. Alpha testing is frequently working for off-the-shelf software as a form of interior approval testing; previous to the software goes to beta testing. Beta testing approaches subsequent to alpha testing. Editions of the software, recognized as beta versions, are unconstrained to a partial audience outside of the programming group. The software is released to collections of people so that additional testing can make sure the creation has little errors or bugs. Occasionally, beta versions are prepared and obtainable to the open public to augment the criticism field to a maximal quantity of prospect users. Lastly, acceptance testing is able to be carried out by the end user, client, or customer to legalize whether or not to recognize the product. Acceptance testing might be carried out as fraction of the hand-off procedure among some two phases of improvement of project. Testing Tools In this section I will discuss a few testing tools that are developed to have a better and enhanced testing process. Program testing and error finding is able to be aided considerably by testing tools and debuggers tools. Kinds of testing/or we can say debug tools such as: Plan monitors, authorizing absolute or partial monitoring of program code comprises: Instruction Set Simulator, permitting whole instruction level examination and outline services Program animation, allowing bit by bit execution and conditional breakpoint at source stage or in machine code code reporting information and report Formatted dump or Symbolic debugging, tools permitting examination of program variables on bugs or at selected points Benchmarks, permitting run-time recital contrasts to be accomplished Performance analysis, or outlining tools that be able to help out to underline hot spots and resource treatment A number of tools feature may be included into an Integrated development environment that also known as IDE. Challenges In this section I will elaborate the different challenges that the Software testing field is facing. Software testing is an experiential examination carried out to offer stakeholders with information regarding the excellence of the product or service beneath test, with esteem to the background in which it is planned to function. This comprises, but is not partial to, the procedure of charring out a program or application with the objective of discovery of software bugs. Testing is competent to never wholly set up the rightness of computer software. Instead, it supplies a condemnation or judgment that contrasts the state and behavior of the result alongside oracles, principles or mechanisms through which somebody might discriminate a trouble. These oracles might comprise stipulations, analogous products, past editions of the similar product, conclusions regarding intended or predictable reason, consumer or client expectations, pertinent standards, proper laws, or other principles (Muccini et al, 2005). Over its continuation, computer software has constant to produce in complication and dimension. Each software product has an intended audience. For instance, the viewers for video game software are fully diverse from banking software. Consequently, when an organization expands or else devotes in a software product, it most probably must charge whether the software product will be satisfactory to its end consumers, its goal audience, its buyers, and other stakeholders. Software testing is the practice of trying to form this evaluation (Polini et al, 2005). A research carried out by NIST in 2002 reports that software bugs cost and the cost to keep up them in the U.S. economy $59.5 billion yearly. More than a third of this expenditure could be evaded if enhanced software testing was performed (Aurum et al, 2006). Software testing is a broad term encompassing a wide spectrum of diverse activities, from the testing of a little piece of code by the developer also called component testing, to the client validation of a large information system that is called acceptance testing, to the monitoring at run-time of a network-centric service-oriented application. In the a variety of stages, the test cases could be planed aiming at diverse objectives, such as revealing deviations from user’s necessities, or reviewing the conformance to a standard arrangement, or assess robustness to demanding load conditions or to malevolent inputs, or measuring specified attributes, such as performance or usability, or approximating the operational dependability, and so on. Besides, the testing action could be carried on according to a prohibited appropriate practice, necessitating rigorous planning and documentation, or somewhat informally and ad hoc can say it exploratory testing. Constructing excellence products carried on to be individual of software development’s furthermost challenges. Testing, one of the majority crucial responsibilities all along the software development life cycle be able to with no trouble go beyond half of a project’s whole attempt. A victorious testing technique is able to accumulate significant effort and increase product superiority, thereby increasing client satisfaction and minimizing maintenance costs (Aurum et al, 2006). Conclusion After analyzing all factors and discussion we can say that without the Software testing process the better working and reliable software is not possible. But in spite of these clear profits, the state of software testing carry out is not as advanced as software development methods in general. Actually, testing practices in industry normally are not extremely sophisticated or effectual. This might be due to the apparent satisfaction from developing something novel as opposed to testing something that previously exists. As well as, many software engineers think testers second-class citizens. They observe testing as a junior or entry position and employ it just as a springboard into development occupations. Though, this awareness might be altering with the introduction of agile development methods, in which testers turn out to be completely integrated members of the development team. Promising approaches have started to find acceptance in business, although the technology relocate among testing research and industry is still inadequate. Academics sometimes articulate that industry is immature and practitioners are clueless, whereas practitioners might argue that researchers waste their time developing cool but ineffective testing technologies. As often happens, the truth lies somewhere in among. Though testing takes a lot of budget of the project development but it provides the assurance of error free working environment. Work Cited 1. Bayse, E. Cavalli, A.R. and Zaidi, F. A passive testing approach based on invariants: application to the wap, Computer Networks, 48(2):235–245, 2005. 2. Beizer, B. Software Testing Techniques, Van Nostrand Reinhold Co., New York, NY, USA, 1990. 3. Berner, S. Weber, R and Keller, R. Observations and lessons learned from automated testing. In Proc. 27th Int. Conf. on Sw. Eng., pages 571–579. ACM, 2005. 4. Bernot, G. Gaudel, M.C. and Marre, B. Software testing based on formal specifications: a theory and a tool, Softw. Eng. J., 6(6):387–405, 1991. 5. Bertolino and Marchetti, E. Software testing, In P. Bourque and R. Dupuis, editors, Guide to the Software Engineering Body of Knowledge SWEBOK, 2004 Version, pages 5–1–5–16. IEEE Computer Society, 2004. http://www.swebok.org. 6. Biffl, S. Aurum, A. Boehm, B. Erdogmus, H and Gruenbacher, P. Value-Based Software Engineering. Springer-Verlag, Heidelberg, Germany, 2006. 7. Briand, L. and Wolf, A. Future of Software Engineering, IEEE-CS Press, 2007. 8. Briand, L.C. Labiche,Y. and Sowka, M. Automated, contract-based user testing of commercial-off-the shelf components, In Proc. 28th Int. Conf. on Sw. Eng., pages 92–101. ACM Press, 2006. 9. Broy, M. Jonsson, B. Katoen, J.P. Leucker, M and Pretschner, A. Model-Based Testing of Reactive Systems - Advanced Lectures, LNCS 3472. Springer Verlag, 2005. 10. Bochmann, G. and Petrenko, A. Protocol testing: review of methods and relevance for software testing. In Proc. ACM/SIGSOFT Int. Symp. Software Testing and Analysis, pages 109–124, 1994. 11. Dardenne, A. van Lamsweerde, and Fickas, S. Goal-Directed Requirements Acquisition, Science of Computing Programming, vol. 20, P.P. 3−50. 1993. 12. Donald C. Gause. Exploring Requirements: Quality Before Design, Dorset House Publishing Company. 1989. 13. Gelperin, D. The Growth of Software Testing, CACM 31 (6). 1988.  14. Giunchiglia. The tropos: Software development methodology processes, models and diagrams, Technical Report No. 0111-20, ITC-IRST. 2001. 15. Hans Van Vliet. Software Engineering: Principles and Practice,John Wiley and Sons. 2002. 16. Heindl, M, Biffl, S. A Case Study on Value based Requirements Tracing, Proceedings of the European Software Engineering Conference and Foundations of Software Engineering, pp.60-69, 2005. 17. Herzog, Pete. Open-Source Security Testing Methodology Manual, at Open Source Security Testing Methodology, (OSSTMM 2.1), 2006. http://www.isecom.org/oml, accessed 23-Oct-2006. 18. Hetzel. The Growth of Software Testing, CACM 31 (6). 1988. 19. Hoboken, NJ. Fundamentals of system engineering, SYS 625 Course Notes. 2005. 20. Ian Sommerville. Software Engineering, Pearson Education (Addison Wesley). 2001. 21. IEEE Standard Glossary of Software Engineering Terminology. quoted at the beginning of Chapter 1: Introduction to the guide, Guide to the Software Engineering Body of Knowledge. Retrieved on 2008-10-28. 22. Kang, K., Kim, S., Lee, J., Kim, K., Shin E., Huh, M. A Feature-Oriented Reuse Method with Domain-Specific Reference Architectures, Annals of Software Engineering, pp.143-168, 1998. 23. Kenneth C. Laudon. Management Information System Sixth Edition, New York. Addison Wesley Publishing Company. 1998. 24. Lee, K., Kang, K., Lee, J. Concepts and Guidelines of Feature Modeling for Product Line Software Engineering, Springer Lecture Notes in Computer Science, Vol. 2319, pp.62-77, 2002. 25. Maiden, N. Gizikis, A. and Robertson, S. Provoking Creativity: Imagine What Your Requirements Could Be Like, IEEE Software, vol. 21, P.P. 68−75. 2004. 26. Mylopoulos, J. M. Understanding ‘Why’ in Software Process Modeling, Analysis, and Design, Proc. 16th Int’l Conf. Software Eng., IEEE CS Press. P.P. 159−168. 2003. 27. Polini, A. and Bertolino. The audition framework for testing web services interoperability. In Proc. EUROMICRO ’05, pages 134–142. IEEE, 2005. 28. Riebisch, M. Supporting Evolutionary Development by Feature Models and Traceability Links, Proceedings of the 11th IEEE International Conference and Workshop on the Engineering of Computer-Based Systems, pp.370 – 377, 2004. 29. Roger S. Pressman. Software Engineering: A Practicioners Approach, McGraw Hill. 2001. 30. Vickers, A. Satisfying Business Problems, IEEE Software, vol. 24, no. 3. P.P.18−20. 2007. 31. Willcocks L., Lester S. The evaluation and management of information systems investments: from feasibility to routine operations, In Information management: the evaluation of information systems investments, Chapman & Hall. 1994. 32. M. Boshernitsan, R. Doong, and A. Savoia. From Daikon to Agitator: lessons and challenges in building a commercial tool for developer testing. In Proc. ACM/SIGSOFT Int. Symp. Software Testing and Analysis, pages 169–180. ACM Press, 2006. 33. Yilmaz, A. M. Porter, A. Krishna, D. Schmidt, Gokhale, A and Natarajan,B. Preserving distributed systems critical properties: a model-driven approach. IEEE Software, 21(6):32–40, 2004. 34. Zhu, H. Hall, P. and May, J. Software unit test coverage and adequacy. ACM Comput. Surv., 29(4):366– 427, 1997. Read More