Society seems to be unwilling to limit complexity because we all want that extra bell, whistle, and feature interaction. Thus, our users always push us to the complexity barrier and how close we can approach that barrier is largely determined by the strength of the techniques we can wield against ever more complex and subtle bugs. Regardless of the limitations, testing is an integral part in software development. It is broadly deployed in every phase in the software development cycle.
Testing is usually performed for the following purposes:. As computers and software are used in critical applications, the outcome of a bug can be severe. Bugs can cause huge losses. Bugs in critical systems have caused airplane crashes, allowed space shuttle missions to go awry, halted trading on the stock market, and worse. Bugs can kill. Bugs can cause disasters. The so-called year Y2K bug has given birth to a cottage industry of consultants and programming tools dedicated to making sure the modern world doesn't come to a screeching halt on the first day of the next century.
Quality means the conformance to the specified design requirement. Being correct, the minimum requirement of quality, means performing as required under specified circumstances. Debugging, a narrow view of software testing, is performed heavily to find out design defects by the programmer. The imperfection of human nature makes it almost impossible to make a moderately complex program correct the first time. Finding the problems and get them fixed [Kaner93] , is the purpose of debugging in programming phase.
Testing can serve as metrics. Testers can make claims based on interpretations of the testing results, which either the product works under certain situations, or it does not work.
We can also compare the quality among different products under the same specification, based on results from the same test. We can not test quality directly, but we can test related factors to make quality visible. Quality has three sets of factors -- functionality, engineering, and adaptability.
These three sets of factors can be thought of as dimensions in the software quality space. Each dimension may be broken down into its component factors and considerations at successively lower levels of detail.
Table 1 illustrates some of the most frequently cited quality considerations. Good testing provides measures for all relevant factors. The importance of any particular factor varies from application to application. Any system where human lives are at stake must place extreme emphasis on reliability and integrity. In the typical business system usability and maintainability are the key factors, while for a one-time scientific program neither may be significant. Our testing, to be fully effective, must be geared to measuring each relevant factor and thus forcing quality to become tangible and visible.
Tests with the purpose of validating the product works are named clean tests, or positive tests. The drawbacks are that it can only validate that the software works for the specified test cases. A finite number of tests can not validate that the software works for all situations. On the contrary, only one failed test is sufficient enough to show that the software does not work.
Dirty tests, or negative tests, refers to the tests aiming at breaking the software, or showing that it does not work. A piece of software must have sufficient exception handling capabilities to survive a significant level of dirty tests. A testable design is a design that can be easily validated, falsified and maintained. Because testing is a rigorous effort and requires significant time and cost, design for testability is also an important design rule for software development.
Software reliability has important relations with many aspects of software, including the structure, and the amount of testing it has been subjected to. Based on an operational profile an estimate of the relative frequency of use of various inputs to the program [Lyu95] , testing can serve as a statistical sampling method to gain failure data for reliability estimation.
Software testing is not mature. It still remains an art, because we still cannot make it a science. We are still using the same testing techniques invented years ago, some of which are crafted methods or heuristics rather than good engineering methods.
Software testing can be costly, but not testing software is even more expensive, especially in places that human lives are at stake. Solving the software-testing problem is no easier than solving the Turing halting problem. We can never be sure that a piece of software is correct. We can never be sure that the specifications are correct.
No verification system can verify every correct program. We can never be certain that a verification system is correct either. There is a plethora of testing methods and testing techniques, serving multiple purposes in different life cycle phases. Classified by purpose, software testing can be divided into: correctness testing, performance testing, reliability testing and security testing.
Classified by life-cycle phase, software testing can be classified into the following categories: requirements phase testing, design phase testing, program phase testing, evaluating test results, installation phase testing, acceptance testing and maintenance testing. By scope, software testing can be categorized as follows: unit testing, component testing, integration testing, and system testing.
Correctness is the minimum requirement of software, the essential purpose of testing. Correctness testing will need some type of oracle, to tell the right behavior from the wrong one.
The tester may or may not know the inside details of the software module under test, e. Therefore, either a white-box point of view or black-box point of view can be taken in testing software. We must note that the black-box and white-box ideas are not limited in correctness testing only. The black-box approach is a testing method in which test data are derived from the specified functional requirements without regard to the final program structure.
Because only the functionality of the software module is of concern, black-box testing also mainly refers to functional testing -- a testing method emphasized on executing the functions and examination of their input and output data. In testing, various inputs are exercised and the outputs are compared against specification to validate the correctness. All test cases are derived from the specification. No implementation details of the code are considered.
It is obvious that the more we have covered in the input space, the more problems we will find and therefore we will be more confident about the quality of the software. Ideally we would be tempted to exhaustively test the input space. But as stated above, exhaustively testing the combinations of valid inputs will be impossible for most of the programs, let alone considering invalid inputs, timing, sequence, and resource variables.
Combinatorial explosion is the major roadblock in functional testing. To make things worse, we can never be sure whether the specification is either correct or complete. Due to limitations of the language used in the specifications usually natural language , ambiguity is often inevitable. Even if we use some type of formal or restricted language, we may still fail to write down all the possible cases in the specification. Sometimes, the specification itself becomes an intractable problem: it is not possible to specify precisely every situation that can be encountered using limited words.
And people can seldom specify clearly what they want -- they usually can tell whether a prototype is, or is not, what they want after they have been finished. Specification problems contributes approximately 30 percent of all bugs in software. The research in black-box testing mainly focuses on how to maximize the effectiveness of testing with minimum cost, usually the number of test cases.
It is not possible to exhaust the input space, but it is possible to exhaustively test a subset of the input space. Partitioning is one of the common techniques. If we have partitioned the input space and assume all the input values in a partition is equivalent, then we only need to test one representative value in each partition to sufficiently cover the whole input space. Domain testing [Beizer95] partitions the input domain into regions, and consider the input values in each domain an equivalent class.
Domains can be exhaustively tested and covered by selecting a representative value s in each domain. Boundary values are of special interest.
Experience shows that test cases that explore boundary conditions have a higher payoff than test cases that do not. Boundary value analysis [Myers79] requires one or more boundary values selected as representative test cases. The difficulties with domain testing are that incorrect domain definitions in the specification can not be efficiently discovered.
Good partitioning requires knowledge of the software structure. A good testing plan will not only contain black-box testing, but also white-box approaches, and combinations of the two. These are as follows:. These are some other useful standards a software tester must know related to QA and software testing. JavaScript Tutorials jQuery Tutorials. Software Testing Standards. Table of Contents. The International Software Testing Standard. It supports dynamic testing, functional and non-functional testing, manual and automated testing, and scripted and unscripted testing.
Risk-based testing is a common industry approach to strategizing and managing testing. Risk-based testing allows testing to be prioritized and focused on the most important features and functions. Part 3: Test documentation. Annex A contains outlines of the contents of each document.
Annex C contains an overview of the examples. Annexes D to S contain examples of the application of the templates. Annex T provides mappings to existing standards.
This standard supports test case design and execution during any phase or type of testing e. IEEE Std specifies a property-independent list of processes, activities and tasks to achieve the claim and show the achievement of the claim.
It provides information to users of the other parts of this International Standard including the combined use of multiple parts. These claims are in the context of assurance for properties of systems and software within life cycle processes for the system or software product. Assurance for a service being operated and managed on an ongoing basis is not covered in this International Standard.
This document focuses on the processes required for successful planning and management of the project's software development effort and for development of the software development plan SDP as a vehicle for representing a project's application of software life cycle processes.
The SDP is a top level technical planning document for a project which addresses technical management processes established by three principal sources the project? This International Standard establishes a common framework for software life cycle processes, with well defined terminology, that can be referenced by the software industry. It contains processes, activities, and tasks that are to be applied during the acquisition of a software system, product or service and during the supply, development, operation, maintenance and disposal of software products.
This is accomplished through the involvement of stakeholders, with the ultimate goal of achieving customer satisfaction. This International Standard applies to the acquisition of software systems, products and services, to the supply, development, operation, maintenance, and disposal of software products and the software portion of any system, whether performed internally or externally to an organization.
Software includes the software portion of firmware. Those aspects of system definition needed to provide the context for software products and services are included.
0コメント