Agile Development and Testing Agile Testing

This section reviews a number of agile development and testing practices that the case studies have highlighted as being particularly valuable from a software quality perspective. I have deliberately not separated the development and testing practices into two separate sections;I believe making this distinction is both artificial and injurious to the cause of developing and delivering quality software.

Iteration–this tried and tested agile development practice provides a powerful means of bringing testing into the development life cycle as early as possible and ensuring testing continues frequently throughout the project. The early identification of defects ensures that the software can be corrected in a timely manner, avoiding the typical problems of late delivery and cost overruns normally associated with late discovery of serious defects.

The downside of iteration, from a testing perspective, is that functional tests and regression test suites grow continuously from iteration to iteration, causing an increasing testing load that if not managed effectively can cause an overload in terms of overall testing effort–particularly toward the end of the project. Use of good test management techniques–such as reuse packs and targeted test execution–can be of value. Test automation through capture–replay tools is another solution that should be considered. Both the practices of identifying targets of test and the use of capture–replay tools are discussed in later sections of this.

Pair Testing

Many of the case studies confirm the positive benefits of the well-established agile practice of pair programming and support the view that, performed correctly, a pair of developers working together is more productive than if both developers were working in isolation. A number of the case studies(that of Thomas, for example) report the need to beware of resistance to this technique by “traditional-style” management on the grounds that it ties up two programming resources whose time might be better spent doing tasks separately.

Analogous to the practice of pair programming, the pairing of test resources with developers has also been reported to be a valuable practice in a number of case studies. The benefits include testers providing valuable quality assurance input to unit test specification, development, and execution, ensuring that developers follow software quality best practices, and acting as a sounding board for developers to discuss development and esting ideas(Thomas, for example). A number of the case studies(Chana, for example) report measurable improvements in the quality of the delivered code using this practice.

As a cautionary note, you must avoid the perception by developers that the testers are there as the Quality Assurance Police (Sewell). One solution is to pair developers and testers but to have the tester review the developers’ work at the end of the day, for example.

Involve Test Resources at All Stages

The practice of involving test resources at all stages is similar to pair testing but ensures that a test practitioner is available at all stages of the development life cycle. Sewell, for example, describes a project in which a dedicated test resource worked alongside the analyst and the customer during the requirements analysis phase, generating test cases directly based on the elicited use cases. This approach was seen to deliver a number of quality benefits:

  • The tester was available to directly advise on the testability of any given requirement(to clarify with the customer how they would verify that a particular requirement had been met by the code during testing, for example), allowing any changes that were necessary to be made while all the relevant stakeholders were still on hand.
  • The tester was able to improve the quality of the requirements by highlighting missing, contradictory, or duplicate requirements, using their quality assurance experience.
  • In parallel with the creation of the use cases, the tester can also review and agree the suitability of the test case design directly with the customer. In practice, involving test resources in the project as early as possible provides a highly effective means of driving testing into the project life cycle at an early stage.

Test-Driven Design and Development

Test-driven design and development(TDD) is a technique that appears to have been widely adopted in the case studies. The success of TDD has also been demonstrated through research(Stapp and Nowakowska). Ensuring that developers specify how they will verify that their code has met its requirements and performed correctly–before they write the code–means that the developer gains a much greater understanding of the requirement and what the corresponding code needs to do to meet it. Since the developer would have needed to have written the test at some point anyway, the overall time taken to complete development and testing of the code is about the same, with the added bonus that the requirements are better understood and implemented, that the test is more closely aligned to the requirement, and that the test is more effective at verifying the correct working of the code. This technique can be particularly effective when used in conjunction with pair testing.

Fix All Defects Immediately

It is a well-established testing fact that finding and fixing defects early in the project life cycle save substantial time and effort, reducing the risk of late detection of bugs, the high cost of correcting them, and the likelihood that the project will miss critical deadlines.

A number of the case studies describe projects that enforce this principle strictly; if defects are detected, development is halted until they are fixed. By using continuous integration techniques plus automated testing, defects are quickly localized to the most recent code change, allowing the developers to quickly focus on correcting the error and getting the project back on track.

In combination with agile practices that allow the team to visualize the real-time status of the project(such as the red and green lava lamp solution that automatically flags a build containing a defect, described by Evans), fixing such a defect becomes a matter of team pride, and there is a determined effort from the team to get the green lava lamp back on again!

Continuous Integration

Continuous integration is another practice that appears to have been very widely used in the case studies. Used in combination with an automated build tool, and incorporating automated unit testing, the code can be automatically tested to quickly identify quality issues associated with that unit.

Where the automated tests fail, following submission of a new component, it is reasonably clear where the fault must lie, and the development process can be halted while the defect is investigated and fixed. Used in combination with some automated and highly visual means of showing that the build is broken, the issue can be quickly highlighted and other project tasks can be suspended while the team concentrates on fixing the defect.

Tilt takes this process a step further and describes a project where automated continuous build practices had been adopted that not only automatically executed unit tests, but also functional and regression tests(implemented in a capture–replay tool), providing very thorough test coverage of the application under development. Wilson further proposes the inclusion of code coverage and code complexity analysis as part of the automated continuous integration and build process.

Test Refactoring

Many case studies report benefits from the established agile practice of code refactoring, particularly where a well-experienced agile team has embraced a culture of joint code ownership. Code refactoring can be particularly successful if conducted in concert with the use of an automatic build tool,which can also execute associated automated unit tests.

Although generally considered a positive technique, Hodgkinson advises that projects should not overrely on code refactoring (particularly when it is used to address design problems), since in his experience this is likely to result in projects failing to make progress.

The use of configuration management tools can also be of significant value in code refactoring, allowing the changes to be undone in a controlled and accurate manner if necessary.

A number of case studies also describe a testing practice that is analogous to code refactoring but involving the tests comprising the test suite. Using test refactoring, test scripts may be changed for the purposes of making them more efficient and/or more effective, or even removed from the test suite(and stored separately for potential future reuse). This process is considered distinct from test maintenance, where a test may need to be changed to reflect some deliberate change to the application under development.

Particular benefits can be gained from using test refactoring on projects where the test suite is very large. Tilt describes a project involving some 35,000 test scripts, where the regression suite took up to ten days to complete. Test refactoring(which included editing existing scripts to make them more efficient, as well as omitting scripts that were no longer needed), in combination with “laserguided testing” (see the next section, Identify Targets of Test), allowed this project to reduce the test suite execution time from ten days down to less than a day!

Test refactoring should be conducted with caution, and tests should only be altered if there is some pressing need for change (such as to make them execute more quickly when overall test suite execution time becomes an issue). As with code refactoring, the use of a configuration management tool is of value to allow changes to the test suite to be undone in a controlled and accurate manner where necessary.

Identify Targets of Test

The practice of identifying the targets of test is of particular value in large-scale projects involving large volumes of test scripts(Tilt, for example), where it may become impractical or even impossible to run or rerun the entire test suite because of lack of time or resources. In such cases, it is necessary to target a specific subset of the overall test suite to execute in the allotted time. Typically, an informal approach is used that is based on the tester’s experience, omitting scripts that address “tried and tested” aspects of the system under test, selecting a subset of the overall set of partition test cases, and making “guestimates” about which parts of the developing system may have been impacted by the most recent code changes.

A risk always exists, when omitting test scripts from a functional or regression test, that apparently innocuous changes to one part of the system under development will have completely unpredictable effects on a seemingly unrelated part of the system, and the decision to omit specific test scripts from a test must be balanced by the potential impact of failing to detect such a situation.

Test automation should be considered as one possible solution to enable large numbers of tests to be executed quickly and accurately(and also to execute overnight or over the weekend to fit additional testing into project timescales using unattendedtesting techniques). Test analysis techniques can be used to provide a formal and rigorous means of selecting subsets of tests to execute and should also be considered.

Code Coverage
Code coverage metrics may also be of value in the process of identifying the targets of test(to make sure that, in selecting a subset of tests to run, you ensure good coverage of the underlying code base). Code coverage tools are also a useful facility to assist you to measure the effectiveness of your testing(and of automated testing tool use).

However, just because you are generating code coverage metrics, do not be lulled into a false sense of security about your code quality;code coverage figures on their own do not improve quality–actions based on them.

Rapid Prototyping

Rapid prototyping is another traditional agile practice (although one that is not without its detractors) that has been highlighted as being of benefit in a number ofcase studies. The generation of simple prototypes prior to generation of full parts of the system under development can have a number of benefits:

  • It helps the developer explore and understand the requirements in more detail.
  • It provides a practical means of verifying the requirements with the customer.
  • It provides a powerful and effective means of obtaining customer feedback on the functionality, look and feel, and usability of the proposed system.

However, in practice, a significant number of prototypes are abandoned, and the actual system is recoded from scratch;in these circumstances, it could be argued that the development effort on the prototype was wasted. Before adopting a development and testing style based on prototyping, the benefits must be weighed against the effort involved in developing additional code that is unlikely to be used in the final system. Another useful approach is to ensure that the prototypes are deliberately developed in such a manner that they can be reused in the final system.

Agile Exploratory Testing

A number of the case studies make a strong argument for the role of manual exploratory testing. Evans, for example, highlights the value of employing a tester to perform the role of a “skeptical” user, creating unusual usage scenarios for the application under test in order to expose defects that automated tools and methods would be unlikely to uncover.

The downside of this approach in an agile project is that it can be an open-ended exercise, with an almost infinite number of possible scenarios that could be explored. Evans’ solution is to engage an experienced and imaginative tester who could fit a set number of effective and efficient tests into the exploratory testing time box. In practice, caution must be taken to ensure a balance; Stephen K. Allott argues that the restrictive timescales in an agile approach to testing may stifle the imagination of exploratory testers and reduce the effectiveness of such testing.

All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd Protection Status

Agile Testing Topics