Agile Process and Project Management Agile Testing

This section reviews a number of agile process and project management practices that the case studies have highlighted as being particularly valuable from a software quality perspective. The role and use of agile meetings are not covered in this section but is given a section of its own due to the importance of meetings in agile projects.

The agile process and project management best practices highlighted by the case studies include the following.

Don’t Throw the Baby Out with the Bathwater

A significant number of the case studies make a very strong case for considering the reuse of traditional project management practices in agile projects:

  • For example, V-model has been cited as having been used successfully to help structure the overall approach to the management of testing ofan agile project(Thompson).
  • Similarly, the use of PRINCE2 has been shown to be of value in the management of agile projects (Denning) in terms of having formally defined and documented roles and responsibilities, having agreed and signed off terms of reference for the project, and having been tasked by a project steering committee to deliver against agreed plans.
  • Formal process improvement schemes, such as the Capability Maturity Model(CMM have also been discussed and employed in a number of case studiesto provide a more formal approach to gaining long-term benefit from the lessons learned on previous agile projects.

It could be argued that agile projects employing these traditional methods are only “agile-esque”– that they haven’t genuinely embraced an agile approach properly, and that the same project could have been completed faster, cheaper, and with better quality using a purist Scrum approach, for example.

However, my personal feeling is that you should make use of whatever tools you find that are able to bring you tangible benefits and that you ignore tried and tested project management solutions at your own peril; if it works for you– reuse it.

Well-Bounded Iterations

Many of the case studies have reported the benefits of arranging the iteration interval around some “naturally” occurring time span, such as a one- or two-week period.

Where it is possible to fit an iteration into such a period, it appears that the natural rhythm of working in a predictable weekly cycle helps to keep the team members focused on their tasks. For example, in Tom Gilb’s case study, developers, testers, managers, and the customer all know that they will be preparing for and performing some specific activities on each day of the week.

One potential downside of this practice is that certain activities(and typically, testing)may be squeezed out of the iteration due to lack of time–for example, where a time-box approach is being used, if the time allocated to development overruns then the time allocated to testing is likely to be eaten into. One suggestion to ensure testing is not missed out is to give the testing task its own separate time box(Kingston).

Agile Estimation

Estimation is a critical aspect of both agile and traditional software development projects. Historically, estimation has been a commonly cited area for concern in traditional projects, with cost overruns and failure to deliver on time or to budget being a common experience.

Although the majority of the case studies did not raise agile estimation as an issue, some notable concerns were raised: Kingston, Warden and May all reported challenges associated with agile estimation.

Warden reports that, on a Scrum project he was involved with, there were consistent problems with estimation(and particularly with accurate completion dates). The metrics and estimates displayed on the burndown chart would appear to be static across the majority of an iteration but would then suddenly slip toward the end of the iteration. Since a key reason for employing Scrum is to quickly identify project slippage, this phenomenon was a cause for concern. This issue appeared to be associated with accurate calculation of velocity; Warden uses the following analogy to illustrate the problem:

The on-board computer in my car has to solve a similar problem to predict the range of the car. It knows how much petrol is left in the tank and the past fuel consumption. The question is how do you use past consumption in a predictor algorithm? Do you take the last 25 miles driving as representative, or 50 or 75? Likewise do you take the last Scrum cycle or use more? Furthermore, how do you normalise story counts so they are comparable across cycles, as stories ranged in size by a factor of 10?

Another issue with agile estimation that Warden identifies is what he terms “sucking the pipe-line dry at the end of a cycle.” To maximize velocity, staff would be allocated to clear as many user stories as possible, including QA team members who would be pulled off testing duties. As a result, although the velocity metrics improved, this approach was storing up problems for later in the project, where there would be reduced timescales and resources, with the inevitable time-scale compromises for testing the software.

In terms of the former estimation issue highlighted by Warden, metrics can play an effective role in addressing the reliable and accurate calculation of velocity. Keeping historical data provides the means of normalizing estimation data and also allows the optimistic and pessimistic reporting styles characteristic of different team members to be recognized. A number of products are available that allow estimated and actual effort to be automatically collated and which can even adjust different practitioners’ estimates based on previous data see [60, 63], for example).

In terms of the latter observation, ensuring that testing plans and resources are ring-fenced is an effective solution to ensuring testing practitioners are not pulled off their quality assurance task and onto project fire-fighting tasks. Approaches used by Sewell, Thomas, and Phillips, for example, have also promoted the idea that having testers involved throughout the project life cycle, and of closer working practices with developers, may actually act to prevent the situation Warden described happening in the first place.

May identifies the difficulties of estimating the time and effort required to produce the code to implement user stories as a challenging exercise, particularly for practitioners new to agile methods. The information was used to calculate sprint velocity and, hence, estimate when the product backlog would be finished. Increasing experience with estimation and the use of metrics collected during previous sprints meant that subsequent estimates became increasingly more accurate. May also cites that keeping the sprints short(of two weeks’ duration in this case study) was of benefit in keeping progress on track.

Finally, Kingston reports issues with estimation on an offshored agile project. In this instance, the problems with compiling accurate progress estimates from the off shored development team was attributed to problems with time zone differences, the difficulty with convening ad hoc offshored meetings, lack of formal cross-geo reporting tools, and the challenge of not having face-to-face communications when the weekly telephone conferences were held. Kingston observes that the use of an agile approach actually highlighted these issues much more acutely than a traditional project approach would have done.

The solutions to these issues that Kingston identifies include improving communications with the offshored team, firmly but politely challenging all estimates, being sensitive to possible language and culture differences, focusing on better communications infrastructure(such as instant messaging and Internet telephony systems), the use of Web 2.0 applications(such as holding virtual meetings in Second Life and videoconferencing technologies, adopting tools that support 24×7 working and process enactment(such as improving the working time overlap across time zones(such as getting one team to start and finish work thirty to sixty minutes earlier each day while the other team starts and finishes thirty to sixty minutes later each day), and, last but not least, getting a member of the offshore team to co-locate with the onshore team to help address language and culture differences(these points are expanded upon in Section).

Progress Measurement

The ability to measure progress in software development projects is a key aspect of their successful management, allowing actual progress against plans to be compared and remedial action taken where problems are identified.

Frequently progress is assessed by collating the estimates of individual progress from the members of the team. However, this approach is notoriously difficult toperform accurately;individual developers and testers may report their own progress optimistically or pessimistically(or may even be “economical with the truth” about what they have or have not done).In small co-located agile projects, this is less likely to be an issue, because frequent contact between the project stakeholders, plus practices such as pair programming, all act to ensure the availability of more accurate progress information.

On larger agile projects, or where the stakeholders are not co-located, this may be an issue. Kingston and Evans for example, describe the challenges of collecting accurate progress estimates from offshore team members where time differences, cultural differences, and just simply the lack of confidence that face-to-face communication provides can cause significant errors in estimating progress. A number of simple solutions can be effective in addressing this issue:

  • Project managers or team leaders should use simple, clear, and unambiguous language when discussing progress and should not be afraid to challenge any estimates with which they are not comfortable. The estimates should be fed back to the team to ensure that the team agree with them.
  • Find other solutions for “face-to-face” communication with offsite or offshore team members such as videoconferencing or webcam technologies, or perhaps by using a tool that can support a virtual meeting, such as Second Life.
  • Ensure you collect metrics on estimates and actuals and use these to “normalize” future estimates(in effect, balancing optimistic and pessimistic reporting styles).
  • Use one of the so-called process enactment tools that ensure developers and testers follow a defined agile process, and where data are automatically collected on planned and actual timescales as the team members plan and executetheir tasks(even automatically adjusting estimates for optimistic and pessimistic workers).

Progress Visualization

On a topic related to progress measurement, a number of the case studies have reported the benefits of publishing or displaying the project progress(often in real time)to the project stakeholders. Examples have included the following:

  • Publish quality metrics, such as the breakdown of where defects are found in the project life cycle(Tilt) to encourage healthy competition between stakeholders in the project(all project metrics were displayed graphically for all stakeholders to see, which was found to be “a great motivator”).
  • Publically display project burndown charts(Warden,; Phillips,; and May,)to ensure all stakeholders are able to review the progress information(but ensure the data are accurate and up to date).
  • Wilson describes a very effective approach in which code complexity and code coverage information are shown to the users as color-coded graphics to display the results of tests executed automatically using his continuous integration, build, and test architecture.
  • Evans describes an automated system where a broken build detected during the continuous integration process would cause a green lava lamp to be turned off and a red lava lamp to come on. It became a matter of significant pride for the team to ensure that the red lava lamp was switched off before the wax could become liquid and start to flow!

Availability of Agile Process Guidance

A number of the case studies stress the importance of providing simple and effective access to agile best practices for the team members. Sewell, for example, describes the benefits of the use of process summary cards from the Essential Unified Process, whereas Hodgkinson and Chana discuss the role of the Rational Unified Process, with its distributed delivery of best practices via a web browser interface.

Although availability of agile best practice guidance on a project is important, the availability of an easily accessed and used source of such information should not be used as a substitute for thorough and effective training in the use of the agile method. Many of the case studies emphasize the importance of well-planned and delivered training in the success of agile projects

In a similar manner, where standard reusable agile templates(such as est script templates)are available for use on a project, these should also be widely accessible and easily customized. Watkins, for example, provides access to downloadable and customizable testing templates, and Appendices E through G of this book provide templates for a number of agile testing artifacts.

Role and Use of Metrics

A number of case studies describe the use of metrics on agile projects and their benefits(Norman; Wilson, Phillips, and Tilt, for example). These can include

  • gaining a quantitative means of assessing the success of some activity–forexample, having the data available to know that the adoption of a particular practice(such as pair testing)has reduced the number of defects detected prior to release of the software by N%;
  • being able to publish or make information available to the other stakeholders to allow them to see the state of the project, to encourage practitioners to improve their performance(perhaps competitively against their colleagues), and to manage the customers expectations;
  • refining the process of estimation. Metrics can be used to compare planned or estimated timescales, cost, and effort with actual figures. This sort of analysis can be used to refine future estimates and plans;
  • being able to report gains in productivity, reductions in cost, and reductions in delivery timescales achieved by adopting a particular agile method to senior management;
  • being able to publish gains in productivity, reductions in cost, and reductions in delivery timescales achieved by adopting a particular agile method to other interested parties in your organization(via newsletters, your intranet, or by lunch and learn presentations);
  • supporting a process improvement program.

while there are undoubtedly very good reasons for setting up and administering a metrics program, there are a number of important things to consider in setting up such a scheme:

  • Make sure you understand why you are interested in metrics–what do you plan to do with the data(such as improving roductivity, reducing defects, and/or process improvement)? This will influence which data you need to target and,
  • Understand your current baseline; you cannot make statements about how your software development process has changed unless you have a quantitative view of where you were before you introduced your agile approach. In practice, this means collecting data as early as is practically possible and requires a great deal of foresight. In some organizations, you may be lucky and data(such as time sheet information)may already be routinely collected.
  • Be inclusive rather than exclusive in terms of the data you collect; later on in the project, it is very difficult go back in time and collect some specific data that you suddenly discover would have been useful to have collected.
  • Collect metrics on the time, effort, and cost of setting up and administering themetrics program
  • you should know how much metrics are costing you.
  • Finally, be aware that the activity of collecting metrics can alter the value of those metrics that you are collecting(known as the Hawthorne Effect).

Watkins provides detailed information on the setup, roll-out, adoption, and ongoing maintenance of a metrics program.

The adoption of a metrics program on an agile project must be balanced by the cost of setting up, administering, and maintaining such a program. Wherever possible, make use of tools that automatically collect, analyze, and report on metrics for you–such as integrated test management tools that can automatically produce reports on test case design versus requirements coverage, for example. Similarly, process enactment products are able to collate developer estimates on both task completion times and the actual values.

Also remember that the activity of collecting metrics itself is not necessarily useful, but that taking action based on the metrics is where the value is obtained(Tilt).

Process Improvement

The ability to fine-tune your agile process, to learn from successes and failures, and to apply the lessons learned to future projects can be of great value. Many case studies(Gilb, Allott, Kingston, Warden, Evans, and Tilt)advocate the use of process improvement in their agile projects, and many of the established agile methods incorporate process improvement in their practices, such as Scrum’s Sprint Retrospectives and Evo’s Key Process Principles.

Metrics is a subject closely related to process improvement; if you don’t have the data to verify that a particular strategy for improving some aspect of your process has been successful, it is difficult to substantiate claims that there have been improvements. If you intend your agile approach to incorporate effective process improvement, you must seriously consider the role metrics can play in such a scheme.

As a final thought, a number of the agile case study authors hold strong views on formal process improvement methods;Gilb, for example, has been heavily involved with the Capability Maturity Model(CMM), and Warden discusses the role of CMM on agile projects in his case study.3 Although such a formal approach to process improvement, involving significant time, effort, and cost to implement, may seem to be at odds with the goals of agile methods, many organizations derive benefits and value from these schemes.

Everyone Is a Software Engineer

It is one of software development’s oldest clich´es that there are frequent project tensions between testers and developers; developers have often perceived testers as

  • the people who were far too meticulous, found defects that the developers had to fix, made delivery of the software late, and made the developers look bad; or
  • the people who didn’t do enough testing, allowed code to be delivered that still contained defects, and made the developers look bad!

As clich´ed as this view may appear to be, there are still many case studies in this that demonstrate that this “us and them” tester–developer issue is still prevalent.

Denning, for example, describes a particularly extreme example of this situation, where the developers and testers were in a virtual war with each other, in which the developers actively looked for opportunities to make the testers look bad to justify their cut of the bonus payment cake!

More enlightened practitioners(including Sewell, and Phillips,)describe agile projects in which the developers, testers, and other stakeholders are all encouraged to take personal responsibility for software quality. Phillips takes this one step further and describes a project where the developer and tester roles are subsumed under the title of software engineer.

Other agile practices can also help challenge these negative prejudices;many of the case studies describe an approach termed pair testing (Sewell, and Kingston, for example). In this practice, developers and testers are paired(in a manner analogous to pair programming) to work together on software quality aspects of the development. This approach appears to have been particularly beneficial;in one such scheme, Thomas reports that developers even began to actively seek out testers to bounce development ideas off them and to get their feedback.

All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd Protection Status

Agile Testing Topics