Determining When to Stop Testing - Essay Example

How to determine when to stop testing Introduction All products are ed to rigid test to determine whether it is now ready to release in the market. Usually tests are done in animals depending on the product for release. This paper is about when is it enough testing and when developers should stop testing their product and release it on the market. The problem is that the customers and the publishers of the product want to release the product as soon as possible and that the product should be as good as it can. However, in order to release a good product on the market or to satisfy a client, deep testing needs to be done in order to insure the quality of the product. The big question here is how much testing needs to be done, because the testing requires time, which is precious because the clients and users want the product as soon as possible. The paper will include sources that will be used in order to support the question of how much testing needs to be done in order to ensure a good quality product and to show that there is a problem when too much or too little testing is done. A review of literature will be presented to show how we come up with the conclusion. The review includes the study done by Rothman (2001) on release criteria. Rothman (2001) tackles when to release a software/ Also in the article written by Barbara Tallent (2008) she discusses when it is enough for testing and releases it in market. At the end of the paper a conclusion was given and the paper will try to find the best answer on the addressed question. Literature Review In the article written by Johanna Rothman (2001) "Release Criteria: Defining the Rules of the Product Release Game," he discusses the problems involve in releasing the product early. He also point out that the decision of releasing the product comes from the higher authority of the organization usually the production managers. The article includes the case of Rita who heads the team in developing software. The production manager is being pressured by the customers and needs to release the product as soon as possible. In order to do so Rita and PM releases criteria for the team to decide. The criterion includes; All code must compile and build for all platforms Zero high priority bugs For all open bugs, documentation in release notes with workarounds All planned QA tests run, at least 90 percent pass. Number of open defects decreasing for the last three weeks. Feature x unit tested by developers, system tested by QA, verified with customers A, B before release. All open defects evaluated by cross-functional team. The decision in releasing the product at earlier time should comply with the release criteria. The release criteria objectives are specific, measurable, attainable, relevant, and trackable. Before releasing the product it should agreed upon by the entire project team and understood by senior management. The reasons for such release should be reasonable enough. Rothman (2001) in the article uses the release criteria to evaluate whether the product can be release or not. This also assesses the entire project and the prediction of an early warning on the incoming problem on the product. Based on the article a product can be released as long as it complies with the release criteria created by the team. Another review on the immature release of the product is shown in the article written by Bernadette Tallent (2008). In this article she presented a case study wherein the VP of marketing for a software company was about to release a new product. The product was incomplete and the team knows it. Extensive user testing on the product has been done and they concluded that the product was missing several key features. Meetings have been done to assess whether or not to release the product in the market. In order to decide in releasing or not releasing the product the group listed trade offs: Reasons to Release The product was stable The product was due to be announced at an Internet World - if we missed the release date we would miss the show The product had been shown to many of our customers and was in demand We had very real revenue pressures for the product The product was brand new with no existing competition - we could be a "first" in the category Reasons not to Release The product was lacking some major features We didn't want to upset our customers by providing them with an incomplete product It wasn't clear we could sell the product without several key features It was potentially disastrous to have the press review an incomplete product Considering their revenue they decided to release the product. After two months they decided to release another product this time with the complete features. The result was the product sold very quickly even the first release. They also receive positive feedbacks from the critics. Based on the case presented in the article they were able to identify when to stop testing. During the work they are working closely with their customers, letting them use the product and give feedbacks. Before releasing the first product the company receives 100 enhancement requests. The company understood the importance of showing the product to their customer and gets their feedback. Tallent (2008) included in her article what important question should be included in testing the product. The most simple and basic question is "Is this product something that can be use today Or "Is this something you would buy today" The writer concluded that if you don't have enough customers for the product yet, then it is better to use absolute minimal feature set to get the product out for people to try. Real customer feedback is ten times more important than the list of features that internal people come up with. New Product Management Several tests are needed to be done in order for the product to release in the market. Among the basic test is the Alpha Test.(Crawford, Di Benedetto, 2006) Alpha Test: Pre-production product testing to find and eliminate the most obvious design defects or deficiencies, usually in a laboratory setting or in some part of the developing firm's regular operations, although in some cases it may be done in controlled settings with lead customers. The Alpha Testing is a crucial "first look" at the initial design, usually done in-house. The results of the Alpha test either confirm that the product performs according to its specifications or uncovers areas where the product is deficient. The testing environment should try to simulate the conditions under which the product will actually be used as closely as possible. The Alpha test should not be performed by the same people who are doing the development work. Since this is the first "flight" for the new product, basic questions of fit and function should be evaluated. Any suggested modifications or revisions to the specifications should be solicited from all parties involved in the evaluation and considered for inclusion. Since the testing is done in-house, special care must be taken to remain as objective as possible. This is the most common testing done on the new product to evaluate its environment or the consumer. If it seems that it satisfies the customer you may have a chance to release the product earlier. Conclusion Testing the product for release in market is important. Identifying the customer and their needs will help the product to become successful in the market. The question is when to stop testing and how to determine it. As what I understand on the review presented, a product may or may not complete the needed test before releasing it in the market. As discussed in the article written by Rothman (2001) as long as the product meets the releasing criteria the product maybe released in the market. The possible problems that may arise during the release should be anticipated by the company and a possible solution on the problem should be deal accordingly. In the article written by Tallent (2008) they proved that product may or may not complete the test but become successful in the market. The team knows that product lacks 3 important features but considering their revenue they need to release the product. Taking the risk the company releases the product in the market showing positive result. To be able to immediately resolve the problem of lacking important features, they created a second product with completed features and recall the first products that are purchased. The team of this company has planned the situation very well. For the company when they satisfy their customer it is now time to stop the testing. For them the customer feedback is ten times more important than list of feature that internal people come up. As a personal view, I think the best critic for the product is the customers. If surveys and questioning as well as actual testing of the product resulted positively together with safety testing of the product then the product is ready to be out in the market. Reference: Crawford, M. and Di Benedetto, A. 2006 The PDMA Glossary for New Product Development, http://www.pdma.org/library/glossary.html Rothman, J. 2001 Release Criteria: Defining the Rules of the Product Release Game http://www.jrothman.com/Papers/ReleaseCriteria.pdf Tallent, B. 2008, When to Release a Software Product , Resources for Product Marketing Managers http://www.infrasystems.com/whentorelease.html Resources: Five Suggestions for the Project Manager - Lay Out Criteria for Important Milestones, and Stick to Them http://www.pmhut.com/ Kaner, C., Rothman, J., 1998 Managing Testing Resources: Five Suggestions for the Project Manager http://www.kaner.com/testres.html Software Program Manager's Network, http://ww.spmn.com The Software Engineering Institute, Risk FAQ, http://www.sei.cmu.edu/organization/programs/sepm/risk/risk.faq.html The Project Management Institute's Project Management Body of Knowledge, http://www.pmi.org. Read More