System Testing - Literature review Example

?System Testing Introduction System testing refers to a process of testing an integrated software and hardware system in order to verify that the system meets its specified requirements. The whole system or major subsystems are evaluated so as to determine their conformance with the required specification (Gomaa, 2011). It is often advisable that system testing be done by an independent team so as to achieve greater objectivity. During the process of system testing various features of the software system need to be tested and these include: functional testing, stress testing, performance testing, configuration testing, security testing and recovery testing (Myers, Sandler, & Badgett, 2011). 1. Functional Testing System functional tests overlap with acceptance tests such that the same tests can apply for both categories. They are both a demonstration of the systems functionality. They are performed to ensure that the behavior of the system is consistent with the requirement specifications set out earlier in software development. All the functional requirements that are set out for the system must be achieved by the system (Burnstein, 2003). A good example is where a financial system is required to allow customers to set up accounts, add, modify, and delete entries into the account, and make printouts. The functional test must ensure that the system can perform these tasks effectively. The functional tests are black box in nature such that the focus is on the inputs and proper outputs for each of the functions that are identified earlier in the requirements. It also takes into consideration improper and illegal inputs which must be tested on the system and their results observed to be consistent with the required functions (Myers, Sandler, & Badgett, 2011). The goals of functional tests should focus on: all types of legal inputs must be accepted by the system; all classes of illegal inputs must be rejected by the system; all the possible classes of system output must be exercised and examined; all effective system states and state transitions must be exercised and examined and lastly all functions must be exercised. It is important that the test results are recorded on a well-defined form and documented (Craig & Jaskiel, 2002). 2. Performance Testing There are two types of requirements for systems: the functional requirement that describes the functions that the software should perform and the quality requirements that describe the quality level expected for the software such as memory use, response time delays etc. The aim of performance tests is to find out if the software meets the performance requirements (Burnstein, 2003). These tests also tell the testers if there are any other hardware factors that might impact on the system’s performance. It allows for testers to tune the system i.e. optimize the allocation of system resources and to predict future performance levels for the system and this is especially important in planning for subsequent releases (Craig & Jaskiel, 2002). It is important that performance objectives be clearly articulated by the user in the requirements document and also in the system test plan. These objectives must be quantifiable. At the end of the testing the tester will know for example, the CPU cycles used, the actual response time in seconds etc. These are then evaluated with reference to the requirement objectives (Myers, Sandler, & Badgett, 2011). Resources for performance testing must be allocated in the system test plan. The resources include the following: a source of transactions to drive the experiments, that is, a load generator; an experimental testbed that includes hardware and software the system-under-test interacts with and these include special laboratory equipment and space that must be reserved for the tests; instrumentation or probes that help to collect the performance data. The probes may be hardware or software in nature; and a set of tools to collect store, process, collect and interpret data (Gomaa, 2011). 3. Stress Testing This is when a system is tested with a load that results in it allocating its resources in maximum amounts. For example when an operating system is required to handle 5 interruptions per second and the load causes 10 interruptions per second then the system is being stressed. The purpose of performing stress tests is to try to break the system and find the circumstances under which it will crash (Burnstein, 2003). Stress testing is very important for any system because it reveals defects in real time as well as weak areas where poor design could lead to unavailability of service. For example, system prioritization orders may not be correct, transaction processing may be poorly designed and therefore lead to memory space wastage and time sequences may not be appropriate for the required tasks. This has been identified as very important for real time systems that often experience unpredictable events that result in input loads that exceed those described in the requirements document (Lewis & Veerapillai, 2004). Stress testing reveals race conditions, deadlocks, depletion of resources in unusual or unplanned patterns and upsets in normal operation of the software system. System limits and threshold values are exercised; hardware and software interactions are also stretched to the limit (Burnstein, 2003). All of these conditions reveal defects and design flaws and these may not be revealed under normal testing conditions. Stress testing is seen as important for the users or client because when the system operates correctly under conditions of stress then clients have confidence that it can perform as required (Lewis & Veerapillai, 2004). 4. Configuration Testing Configuration testing allows software developers to evaluate systems performance and availability when hardware exchanges and reconfiguration occurs (Burnstein, 2003). Normally software systems interact with hardware devices such as printers and disk drives. Others also interact with multiple CPU’s, some of which are redundant. Software that controls real-time processes also interfaces with devices but these are highly specialized hardware such as missile launchers or nuclear power device sensors (Burnstein, 2003). May users or clients will require that devices be interchangeable, removable or reconfigurable. Configuration testing requires many resources including multiple hardware devices used for tests. If a system is found not to have specific requirements for device configuration changes then large scale configuration testing is not essential (Gomaa, 2011). Various objectives have been identified for performing configuration testing and these include: to show that all the configuration changing commands and menus work as expected; to show that all interchangeable devices are really interchangeable, and that they each enter the proper states for the specified conditions; and to show that he systems’ performance level is maintained when devices are interchangeable or when they fail (Craig & Jaskiel, 2002). Performance testing should incorporate many operations and these include: rotating and permutating the positions of devices to ensure physical/ logical device permutations work for each device; inducing malfunctions in each of the devices to find out if the system properly handles the malfunction; and inducing multiple device malfunctions to see how the system reacts (Myers, Sandler, & Badgett, 2011). The objective of these operations is to reveal defects that relates to hardware or software interactions when hardware exchanges and reconfigurations occur. The consequences of these actions are observed to determine whether the system can recover gracefully in case of a malfunction (Burnstein, 2003). 5. Security Testing Security testing evaluates system characteristics that relates to the availability, integrity and confidentiality of system data and services (Burnstein, 2003). Many developers face the problem of ensuring that the system is safe and secure and this is especially true because of the proliferation of commercial applications for use on the internet. The initial point is for clients to specify clearly in the requirements so that their security issues can be addressed by designers and testers (Lewis & Veerapillai, 2004). Software and data can be compromised by: criminals’ intent on doing damage, stealing data and information, causing denial of service, invading privacy etc.; errors by developers who modify, destroy or compromise data due to misinformation, misunderstandings and lack of knowledge (Craig & Jaskiel, 2002). All this can be done by individuals in or outside the organization and can be random or systematic. Damage can be done through various means such as, viruses, Trojans, trap doors and illicit channels. Developers ensure the security of systems through use of protection mechanisms such as passwords, encryption, virus checkers, and the elimination of trap doors (Burnstein, 2003). Security testing is done through password checking and a number of other tests. In password testing the designers need to answer length required for password, characters allowed for passwords etc. the following tests are also performed: password checking; checking legal and illegal entry with passwords; testing password expiration; testing encryption and decryption algorithms for systems with encoded data; evaluating browsing privileges; identify trap doors i.e. unprotected entries into the system and testing for virus checkers. Other tests may involve use of a test group that will attack the system from various angles to determine security (Burnstein, 2003). 6. Recovery Testing Recovery testing is done by subjecting a system to losses of resources in order to determine if the system will recover properly from the losses it has been subjected to (Burnstein, 2003). Recovery testing is important for systems such as transactional systems e.g. online banking software. Tests would determine if the system could return to a well-known state without any of the data being compromised. Automated systems are designed in this way; they have multiple CPU’s and mechanisms to detect device failure; and checkpoint systems which records transactions periodically and preserves them in case of system failure. The records ensure that the systems return to normalcy in case of failure. Recovery testers must ensure that the device monitoring systems and checkpoint software are working properly (Watkins & Mills, 2010). Testers should therefore focus on the following: Restart: the current system state and transactions are discarded. The most recent checkpoint record is retrieved and the system initialized to the states in the checkpoint record. The work of testers is to ensure that the transactions have been reconstructed correctly and that all devices are in proper working condition (Watkins & Mills, 2010). Switchover: the ability of the system to switch to a new processor must be tested. Switchover is the result of a command or a detection of faulty processor by a monitor (Watkins & Mills, 2010). In the above situations, all transactions and processes must be carefully examined and to detect: loss of transactions, merging of transactions, incorrect transactions, and an unnecessary duplication of a transaction. A good way to know if these problems exist is to perform recovery testing under a stressful load. Transactional inaccuracies and system crashes are likely to occur with the result that defects and design flaws will be revealed (Burnstein, 2003). References Burnstein, I. (2003). Practical software testing: a process-oriented approach. New York: Springer. Craig, R. D., & Jaskiel, S. P. (2002). Systematic software testing. Norwood: Artech House Publishing. Gomaa, H. (2011). Software Modelling and Design: UML,Use cases, Patterns, and Software Architectures. Cambridge, UK: Cambridge University Press. Lewis, W. E., & Veerapillai, G. (2004). Software Testing and Continuous Quality improvement. Boca raton, Florida: Auerbach Publications. Myers, G. J., Sandler, C., & Badgett, T. (2011). The Art of Software Testing. New York: John Wiley and Sons. Watkins, J., & Mills, S. (2010). Testing IT: An Off-the -shekf software Testing process. Cambridge: Cambridge University press. Read More