Testing A Derivatives Trading System In An Uncooperative Environment Introduction Agile Testing

My name is Nick Denning and at the time of this case study I was the managing director of Strategic Thought Limited, a company I founded in 1987, which we subsequently floated on AIM1 in 2005 and which now specializes in the development and sale of financial risk management products. At the time the events in this case study took place, we were an IT consultancy with a strong reputation for work in the Ingres Relational Database Management System arena. A particular strength of ours was to be able to analyze systems, identify performance issues, and make changes to the database configuration, the database structures and the SQL code within the application.

Making changes to mission-critical applications,sometimes directly on the production environment, required a careful consideration of the implications for a change and the ability to assess the risks and agree with a customer on the risk reward profile to justify implementing changes that bypassed the conventional release procedures.

Our successful approach to this problem resulted in a number of engagements to assist customers in building regression test environments so that they could continue to design, implement and deploy changes to a solution quickly and confidently on completion of the engagement with us. A significant class of problems were not functional but rather performance related;solutions did not scale up underload.They might simply slow down and be unusable, and in the worst case they might run out of resources and crash.Our regression test approach began to incorporate a “performance pack” with which we ran tests on duplicate installations, using copies of production data and examples of transactions run a few days earlier in production, to identify the reasons for solutions failing.These projects then led to engagements associated with capacity planning to establish the upper limits for throughput on systems, such as prior to the deregulation of markets when a major increase in business was expected.

This case study describes a project where we were engaged to help our customer improve the reliability and availability of a mission-critical system. The case study describes an all-too-familiar problem in the software industry; the development of a business-critical software solution, initially built by an in-house team, with a focus on the delivery of key business functionality, to very tight timescales,but which after two years of development displayed serious quality problems during day-to-day use.

The quality issues were found in two main areas:

  1. Functionally the solution was known to have a number of bugs within it when delivered, but generally it was possible for the back-office users to work around these issues.In practice, it was found that only experienced users could perform the work-around, and it took those users longer to use the system than had been expected.A dominating factor was a business desire for new capabilities to be delivered urgently, resulting in staff being rewarded by meeting delivery deadlines,resulting in an informal policy of deploying into production comewhat-may and fixing bugs in production.As a consequence insufficient effort was put into testing prior to release.
  2. Reliability and availability were also the issues; under circumstances of heavy load,or indeed unexpectedly and without warning,the solution could crash, after which it could take a number of hours to recover.

Given the business-critical nature of the solution,combined with the timecritical nature of the derivatives market, these quality issues were unacceptable; unless options (a particular financial instrument) could be exercised at precisely the right time, it would not be possible to realize the profits earned through those options.

This was not a theoretical risk–in fact this worst-case scenario had already manifested itself;the customer had lost substantial amounts of revenue because the system had crashed on these critical “end-of-period” dates, and it had not been possible to recover the system and exercise these options.Although difficult to quantify precisely,events of this nature could,in the worst case, cost the company in excess of hundreds of thousands of pounds (which does not include the intangible but important effect of loss of reputation). This event, and the likelihood that it would occur again in the future, provided the customer with a compelling reason to investigate how the problem could be solved.

Strategic Thought was approached by the customer to assist them in the development and testing of the next phase of this project. The next section reviews the testing challenge facing our team.


All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd DMCA.com Protection Status

Agile Testing Topics