Why Testing is Difficult - Testing Tools

Consider a simple example: to test a program which takes three numbers (for three sides of a triangle) as input and checks whether the sides form a triangle or not [Meyer 1979]. How do we tlst the program? We have to give different combinations of inputs (3 numbers) and check whether the program is working correctly. We need to give correct inputs (numbers which form a triangle) and wrong inputs (numbers which do not form a triangle) and ensure that the output is as expected. To give the correct numbers again, the numbers can be integers, floating point numbers or a combination of both. We also need to give characters and check the program (if you give single characters, it gives interesting output!). We need to give small integers (say, 10,12,3) and also big integers (568888,9999999 etc). We need to give some inputs that have one or two 0's,and all zeros and check how the program is functioning. So, to carry out an exhaustive testing of such a simple problem, the input data to be given will be enormous. It takes nearly 2 hours to give all possible combinations and make sure that this program is ok. Testing is difficult because:

• We need to test the software for both right and wrong inputs and check the functionality as well as the performance parameters.
• We need to give the inputs in such a way that each and every line of the code is tested.
• We need to give inputs randomly and check that the software never fails.
• We need to test the software as though a normal user is using it and check whether the necessary error messages, help etc. are provided.
• We need to test the software by simulating the actual environment. For example, if a database application has to be accessed by 100 users simultaneously, it is not enough if you test the software for 2 or 10 users and declare that the software is working fine.
• In many cases, it is not possible to test the software in actual environment. For example, how do you test the software used in a satellite launch vehicle or a missile? You need to do the entire testing only in a simulation environment.

As the complexity of the software increases, the time taken to do such exhaustive testing becomes enormous and exhaustive testing becomes humanly impossible. To summarize, testing is difficult! To carry out the testing in a limited time with limited effort and resources, and try to bring out all the possible hidden defects is a challenging and intelligent task.