Lessons Learned The Payoffs And Perils Of Offshored Agile Projects - Agile Testing

There were a number of lessons learned from our agile offshored project:

  • Fundamentally, offshoring is a trade-off between speed and cost of development -This may not be such an obvious issue on a project employing traditional development and testing approaches, but the use of an agile approach did highlightthe problems. For example, our feeling is that we could have run to shorter oneweek iterations(with which we have had very positive experiences of in previous projects)if we had not had the overhead of long-distance communications to cope with. Instead, we had to adopt a two-week-iteration period, which meant that the team did not get into quite such a slick rhythm and were of necessity slightly less productive.
  • Reduction in productivity– Now, the key issue is – was the reduction in productivity offset by the reduction in the unit cost of the offshored resources? In this project, our feeling(justified by metrics)was yes – in fact, overall we believe the project was some 30% cheaper than if we had based the work solely in the United Kingdom(and employed a co-located team).
  • Software quality lessons– There were some very valuable software quality lessons to come out of this project:
  • Pair testing– We definitely recommend the use of this approach. Metrics showed that when two testers worked together to design and implement tests, they
  • showed a greater understanding of the underlying requirements,
  • identified more and better data points(to test the correct operation of the code, as well as in terms of error and exception values),
  • were more productive(producing some 15% more tests per unit time than\two testers working in isolation), and
  • lismiled more – this social activity made for happier and more motivated testers.
  • Test-driven development(TDD)– The feedback from the developers is that TDD is a very valuable approach. The need to understand the requirements in sufficient detail to be able to design a test to show that the code meets the requirement before implementing the code means that the developers have gained a thorough understanding of the intention of the requirement before they come to code it. Some developers expressed the view that TDD would slow them down and reduce their productivity;in practice, there has been no appreciable reduction in developer productivity, but there has been a measurable improvement in the quality of the code(in terms of matching requirements more closely as well as in delivery of code with fewer defects in it).
  • Test automation– The use of the JUnit testing tool has been an unqualified success and we would definitely use this again on similar projects. The results of the use of the capture–replay tool have been mixed:
  • There is a definite benefit in software quality from being able to rerun tests recorded in earlier iterations(we make sure that old defects don’t reappear, for example).
  • Also, the more often you replay a given test script, the more return you gain on your investment. This s important, because these tools are a significant project investment(both in the cost of the tool plus staff training/learning time). We estimate that we needed to reuse a script four or more times to realize a gain in productivity versus an equivalent manual test. So for a short one-off project you need to challenge the use of these tools. Where you are likely to gain good reuse from the tools, there are economic gains to be made.
  • A good lesson to learn is that, to use these tools effectively, you must ensure you either have testers who have used the tools in the past or ensure that you budget time and money for training.
  • As a final thought, if your code base or application functionality changes too frequently between iterations, you may also incur additional cost from maintaining the test scripts. Although this sort of project chaos should clearly be discouraged by the project manager, script maintenance costs must be balanced against your return-on-investment metrics.
  • Requirements coverage– If you do not know what the software you are developing is supposed to do, how can you possibly test it correctly? For any nontrivial project, the use of a requirements management tool is essential. The ability to elicit, document, and maintain the requirements for the system enables developers to know what they are developing, as well as allowing the testers to know what it is they are testing. In particular, the use of a requirements tool should allow the test manager to be able to make some informed estimates about testing timescales and resources and allow the test analyst to demonstrate complete test coverage of the requirements. A further bonus of using a requirements tool is the ability for remote stakeholders to use the product’s web-based interface to inspect, comment on, and request changes to the requirements.
  • Testers beware of time boxing– Time boxing is clearly a valuable tool in the project manager’s arsenal to help keep development on track and to achieve delivery deadlines. However, during a couple of our iterations, the time allocated in the plan for testing was eaten into by overrunning development. If you plan to use a time-boxed approach, you must ensure that the time planned for testing is sacrosanct. We considered, but in the end didn’t implement, making the functional testing activity a time box of its own that followed on immediately from the associated development time box. This is definitely something I would investigate in future projects.
  • Communications issuesWe learned some very valuable lessons about communications, and how good and bad communications can affect the choice of using an agile approach:
  • Lack of face-to-face communication matters. It is difficult to build a rapport with someone on the other side of the world just using a telephone. As a result you lose some very valuable face-to-face cues, such as the confidence with which an estimate to complete is delivered during a telephone progress meeting. A true agile approach would of course have mandated the co-location of the stakeholders. One method that I have used that worked particularly well was to have one of the offshore team work onshore as a liaison(or even as translator). They are able to deal with people at both ends of the telephone and are often well placed to explain what the other team really means.
  • Find strategies to build a trusted relationship with offshored colleagues. Find the time to engage in small talk, find out about their interests and hobbies, discuss family stuff, and even just talk about the weather. All parties need to find the means to understand and trust each other. Having an offshore team member working onshore is also a good way of building a trusting relationship with the remote group. However, it is important to make this person feel like a full member of the team and to include them in all activities, bearing in mind there are cultural differences. For example, one Indian colleague revealed, but only through general conversation, that three days earlier he had become a father for the first time but didn’t think it polite to mention the fact. We were able to discuss cultural mores while “wetting the baby’s head” before presenting him with some cuddly toys and packing him off home to meet his daughter.
  • Be polite, but firmly challenge all estimates. It is all too easy for someone on the end of a telephone(and themselves under project pressures)to provide overly optimistic estimates of their progress. This is particularly true when the communications are infrequent–a team member may actually give an estimate that is too optimistic, believing that by he next time you were in communication, they could have brought a late task back on track. The best solution(and one we would look to use in similar future projects)would be to use an agile distributed development tool(such as in)where individual worker’s estimates and actuals are automatically collected as they perform the work and which are instantly available to the project manager to review.
  • Be aware of language/meaning issues. For example, one of the Indian offshore practitioners, when asked how a particular delivery had gone, used to say “ship happens”; over the telephone and with the unfamiliar accent, this message was interpreted in a “negative manner” as something bad had happened, rather than in the positive manner it was meant to have been interpreted(i.e., that the software had in fact shipped).
  • Funnily enough, we had similar problems with our U.S. colleague. In one transatlantic teleconference, our project manager wanted to highlight what he thought was a key requirement and said something like, “let’s place this issue on the table” (meaning “let’s focus on it”). At the next weekly meeting, our project manager was surprised to find that no progress had been made on the requirement; it turns out that in the United States, to put something “on the table” means to put it aside and to focus on other issues!
  • My final, and perhaps most interesting example, however, is of one offshore practitioner who was working onshore in the United Kingdom. He was perfectly polite when arranging meetings but would always ask to meet people “on” their convenience.
  • Find better means of more frequent communication.
  • Make use of instant messagingtechnologies to keep in touch. Such technologies can be very useful to ask simple spontaneous questions or to seek clarification(but beware of time zone differences).
  • Consider the use of cheap Internet telephony. Although this may not be too useful for offshore sites where there is no working time-zone overlap, it can be a powerful adjunct to instant messaging, allowing workers to have cheap and easily available voice calls.
  • Consider the role and use of videoconferencing. Although not so much in vogue as it was in the 1990s and early 2000s, the face-to-face aspects provided by a video link for holding offshore meetings might well provide a better means of obtaining subtle cues from body language and expressions that straightforward telephone calls can’t.
  • Consider the use of Web 2.0 social networking technologies. For example, consider having on-line virtual meetings in Second Life. This may well be an approach that improves team cohesion and encourages effective communication.
  • Ensure you only select tools that are distributed. For example, the requirements management tool we selected included a web-based interface that permitted our offshore colleagues(and, in particular, the U.S.-based marketing manager) to log on to the tool remotely, review the current requirements, and, where necessary, make comments or request changes to the requirements.
  • Make more use of tools that support 24/7 working– Having tools that automatically replicate the code base and project data between geographically distributed sites would be a very valuable means of supporting the distributed working model we were looking to achieve. We discovered a very powerful approach that would have automatically replicated our project workspace between sites just after the project completed and we would definitely investigate its use in future projects of this nature.
  • Make use of process enactment tools. In running an agile approach to developing the new software, it would be of value to have all of the practitioners follow the same process. Taken a step further, it would be nice if all the practitioners were constrained to follow the same process. There are products available that provide powerful facilities to implement and enact (enforce) particular style of project management(such as agile) and which will also produce accurate real-time reports and dashboard displays of project progress. Employing a distributed tool of this nature would be a powerful solution to drive an agile offshore project more effectively.
  • Try to harmonize working times across time zones. As described earlier, over the ten-week project duration, we overran by two weeks. This slippage was largely caused by the problems of “round-trip communications” (i.e., where a request from the U.K. team – email or voicemail – was made outside of the Indian working day, picked up by the Indian team the next day, worked on, and then reported back to the U.K. team the next day). As the project proceeded, we did try to find solutions to this phenomenon – such as making sure we were in touch with the Indian team during their working period, and changing our respective start and stop times to improve overlap(i.e., asking one or more of the Indian team to start work an hour later than usual and finish work an hour later and doing the converse with the U.K. team). This certainly helped and meant that, toward the end of the project, round trip communication became less and less of an issue.

All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd DMCA.com Protection Status

Agile Testing Topics