Testing Approaches - Testing Tools

In a large software project, it is impractical to integrate all the modules in one shot and start testing the software as a whole. The system has to be built in stages and the product has to be built incrementally carrying out testing on each incremented software. Such a systematic approach helps in easier debugging. The testing approaches discussed in this section help in achieving this objective.

Top-down Approach versus Bottom-up Approach

Top-down approach:

In this approach, testing is done from top of hierarchy. Dummy routines called studs that simulate a module are introduced.

Bottom-up approach:

In this approach, testing is done from bottom of hierarchy. Dummy routines called drivers are introduced that invoke the module.

Consider testing the software that implements a stack. The program contains a main function and functions to push and pop. In top-down approach, first the main function is tested. For this, in the place where push function is called, a print statement can be introduced which prints "call the push function". Similarly, at the point where pop function is called, a print statement which prints "pop function is called" can be introduced. In bottom-up approach, a driver function is written, which just calls the push function first and then testing is done to ensure that the push works well. Similarly, pop function is tested. Subsequently, all the functions are combined together and tested.

Functional Testing versus Structural Testing

In functional testing, functionality of the module is tested and structure is not considered. Test cases based on specifications and internals of modules are not considered. This type of testing is also known as black box testing.

Structural testing is used to test the implementation of the program. Here, the source code is looked into for testing the software. Structural testing involves (a) statement coverage (b) branch coverage and (c) path coverage.

Statement coverage:

This ensures that each and every statement is tested. Software tools called "profilers" are used to carry out this statement coverage testing. Profilers indicate the number of times each line in the code is executed.

Branch coverage:

In branch coverage, each and every condition is taken, and inputs are given in such a way that each branch is executed at least once.

E.g. if (a > 6 && b < 5)

To test this branch, inputs for a and b should be given in such a way that a > 6 is true and false; b < 5 is true and false.

Path coverage:

To test loops, this is required. For example, the loop statement for(i =0; i <= 100; i++) is executed 101 times. Does the programmer really want 101 times or only 100 times? Invariably programmers make mistakes at the boundary values. So, testing has to be done at loop boundaries.

Mutation Testing

Mutation testing is required to ensure that the software does not fail. It is also a good debugging mechanism. After the software works correctly, mutation testing can be done to simulate wrong inputs. In mutation testing, program is modified (or logic is changed) slightly to obtain mutants of the program. Different mutants are tested with the same test cases. If the mutant fails, and the actual program works correctly, confidence is gained in the program more and test cases are considered as good i.e., different mutants should give different results. Generally each mutant will have only one change. To produce mutants, mutation operators are defined. The mutation operators can be Constant replacement Variable replacement Arithmetic operator replacement Relational operator replacement Goto label replacement Consider the following code segment: input y; x = sin(y); print x; When you execute this program, you input a value for y and then check whether the value of sin (y) is displayed. If correct result is not obtained, a simple way of debugging the program is to create a mutant of the program. You can create the mutant by modifying the program as follows:

input y; x = y; print x;

Here, we removed the sine function. Now, when you execute the program, if the result is correct, we can isolate where the problem is: it is in the sine calculation. The mistake that is commonly done is not to give the value of the argument y in radians for sine calculation.

Regression Testing

When a defect is reported in the software, the developer makes some change to the software to remove that defect. It is likely that the change made in the code may lead to another defect that may not be visible immediately. So, whenever a change is made to the source code, one has to ensure that there are no ill effects of the change on the other parts of the software. Regression testing is done precisely to ensure that change in one part of the software has no ill effect on other parts of the software.

Whenever a change is made to the source code, a set of pre-defined test cases has to be run to check whether any other portion of the software is affected.

For example, consider the following code segment: input x;

y = x ** 2; print y; if (y == 100) print "Alarm: x value is 10";

Initially,you wrote the program which takes a number as input, squares the number and checks whether the square of the number is 100 or not. If it is 100, it prints an alarm. Suppose later on, you change the logic and calculate the value of y as cube of x instead of square of x. You may simply change the second line as:

y = x*; x;

Suppose you now test your program by giving some inputs 2,6 and 8; and find that the program is working fine. Strictly, the program is not working fine, because you also need to change the if statement as follows: if (y== 1000)

So, it is very important that even if you change one line of code, you need to test the software again thoroughly. This re-testing is called regression testing.


All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd DMCA.com Protection Status

Testing Tools Topics