The COTS Integration Testing Challenge Agile Testing

The COTS system purchased by our client was a sophisticated and special-purpose piece of software that was designed to be installed on a server and to be controlled by a separate bespoke application. In our case, the bespoke application was the existing order management system. The objective of the project was to adapt this application to make it work with the COTS system in a way that met the stated business requirements.The task involved extending its user interface, writing control logic and integrating with various legacy systems.

System requirements for the project were defined by performing a combination of top-down(business-driven) and bottom-up (technology-driven) analysis. In many cases, the business processes had to be changed to fit in with the way that the COTS system operated,which resulted in a significant amount of functional change to the existing bespoke system.

In order to reduce project risk,we followed an agile approach to software engineering that we have developed and refined through its use on many projects. Our approach is based on the Rational Unified Process(RUP)framework. It adopts many agile practices,including early development and short(two-to three-week) iterations, with each iteration combining analysis, development and testing activities. It also recognizes the importance of software architecture in attaining consistency and resilience to change,but at the same time developers are allowed to work unconstrained within their allocated unit of work. Agile development techniques such as pair programming and test-driven development were also adopted.

The implication of this way of working, however, is that there will always be a certain amount of change all the way through the project. Good change management was therefore critically important, as was the ability to understand the state of development at any point in time. The project manager needed to be confident about which requirements were implemented and working, irrespective of any other requirements that may have changed.

One practical way of achieving this was through automated regression testing.Such tests can be written at the unit or integration level(or preferably a combination of the two). However, due to project constraints, it became apparent to us that we would be limited in the number of automated integration tests that could be written and maintained.

Specifically, this limitation was caused by the difficulty of writing integration tests that were sufficiently focused on individual software requirements. In terms of code coverage, a single integration test would typically exercise a very large part of the system and consequently, test many requirements. This was initially seen as a good thing but, as the test base expanded, we found that some parts of the system were being tested many times by many different tests.This made those parts of the system very difficult to change without also modifying every related test. In addition, integration tests(even automated ones) can become quite slow to execute, which made it disruptive for our developers to run these tests on a regular basis(for example, before committing changes to the source control repository).

A partial solution to this problem was to focus developer testing on the writing of automated unit tests. Execution of these tests by the developers before committing code was made a mandatory part of the development process. Once code had been committed, it was then automatically built and tested a second time by a continuous build server that would send out email notifications in the event of a failure.As such, a test failure could never remain undetected for more than a few minutes – a factor that vastly reduced the amount of time required for detecting and fixing any related bugs.

A small number of automated integration tests were retained as a sanity check, but the bulk of the integration testing was performed manually by developers at the point where each build was prepared for release to system testing.


All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd DMCA.com Protection Status

Agile Testing Topics