|
Oct
09 |
Topic: No Tags
Going down the path of off shoring testing (be it the definition of test cases, functional testing, or even automation of regression testing) will cause the team to fail their Agile adoption. First off, I should probably define what I mean by “Professional Testing” – in this case what I mean is software testing performed and managed by individuals whose full time job is software testing and who have been trained in effective testing techniques and tools. Many teams find themselves in a situation where they do not have this perfect scenario. In fact, there are numerous teams who don’t have professional testers at all. It is common that these teams use business analysts, the product owner and/or developers to perform all levels of tests: unit testing, functional testing, system testing, regression testing, acceptance testing, integration testing, performance testing, etc… Generally, in these situations, the individuals performing and managing the testing have not been trained at all on testing techniques, best practices, or testing tools. In this situation, the team and management usually understand the importance of professional testers and their value, but are unable to staff their team due to the lack of availability of professional testers within the organization or the lack of funds to obtain professional testers. Therefore, the team and management start looking for ways to gain the benefit of professional testers in other ways. It is during this search where off shoring some of the testing often comes up. On the face of the suggestion, it looks promising. The team can get support from the professional testers that they need but at a price tag which is palatable to the business’s budget. Everyone wins, right? Well, let’s see… Testing is one of the crucial feedback layers within Agile. Developers and testers must work very closely together to clearly define what is being built and the expectations of the result. Developers and testers should hear the requests and requirements for work at the same time from the same people to provide consistency as well as an ability to check each other, promoting a clear understanding of what was requested. Additionally, Agile testers must test from the perspective of the users who will utilize the system and this perspective and understanding is established and reinforced by high levels of collaboration and communication with the users. If your testers are separated by thousands of miles and a multitude of time zones from your team as well as your users, the collaboration that is required is close to impossible. Open communication between these groups will be attempted, but schedules will not overlap, conversations will be cut short because of the time of the day, and misunderstandings will abound due to limited real-time communication. Clarifications around details or misunderstandings will now take a day to get since the testers can only send an email and then wait until the beginning of their next day to get the answer. Soon reams of requirements documents will be produced and used to define what the testers should be testing. Because of the need to produce the big documents, the analysts need more time to gather and document the detailed requirements. Then, the testers need time to read through and understand these huge requirements documents. In the meantime the developers have been involved in these requirements discussions with the users and have already been developing well before the testers get their understanding of the requirements (often even before they even get the documents). In fact, what the team finds is that development is always done before the testers are ready to begin. And when this development is done, will the developers sit idly and patiently await for the test results to them fix bugs, or will they move on to new development tasks, further complicating the problem? So, let’s summarize: We now have an Agile team which has a requirements gathering and document writing phase that must precede the actual development and testing. We also have development that is fully complete prior to starting testing. Sounds a little like Niagra Falls, doesn’t it? This team will get few benefits from other aspects of agile that it is able to implement- Agile is a holistic thing- the best benefits are realized when you do it all. This may seem a little extreme, but it does happen on one level or another. Can we just automate the regression with offshore testers? Is there any other way? Thoughts?
Trackback URL: http://www.bigvisible.com/dladd/is-offshore-testing-worse-than-no-professional-testing/trackback/
|
||||||
No, I don’t see Offshore Testing Worse Than No Professional Testing.
Off shoring is the business demand and very cost effective profitable business for the organization.
Very true, testers are separated by thousands of miles and a multitude of time zones from your team as well as your users. But the collaboration that is required is not at all close to impossible, of course at initial stage expectation could not be matched immediately as of tasks perform at parent location.
Once you define the process, keep all the requirements handy and up-to-date and do constant and continual monitoring of execution, will help to achieve the desired goal.
Why can’t regression test be automated by offshore testers? These testers might not be expert in the application business domain but well versed in writing and maintaining the code/scripts to automate the tasks to do. Gradually with time and experience tester will know the application.
I work without having enough professional testing experts in the team so what could be the way? Nothing to worry much, make the team well versed with the defined process of testing process to follow and bring them to that level. Of course, there will be a senior tester/team member with required skill to bring other team member to the level of tasks execution.
For the most part I agree with you. OTOH, if you are defining acceptance test criteria, then it’s entirely possible you might be able to use that as a means to offshore some UI level regression suite work based on those acceptance tests. You wouldn’t want that to be your entire regression suite, then then you likely don’t want to have that working all the way up at the UI level anyway, but rather at a ’skinned’ level just below the UI so the tests execute faster and are less brittle.
You’d very-likely need to on-shore at least one person in that team for a little while to teach them the app etc (which will increase the cost) and give them a bit of domain knowlege, so they understand the acceptance tests.
I’ve done something similar to that with a tester in another office. We define acceptance tests using a BDD style format (given, when, then) and he then automates those for us using a combination of Watir, Watircraft framework, Cucumber (it’s a web UI). This means our actual test code is written ruby, but that’s ok as it’s an easy langage to teach to QA folks, and it reads clear enough that our devs (working in asp/c#/.net) can still make sense of them if need be.
BUT the key to that success is that we’re leveredging the BDD work already done to create those acceptance test stories using the DSL and BDD format, which are used by both dev and QA during the development cycle.. So we’re not creating extra documentation etc.
That being said however, I’m doing this ‘nearshore’ working between our office in Seattle area, and a Bay Area office where we have a tester with some extra bandwidth, and who was eager to learn automation so it could also be applied to the products being produced in that office. So when he had questions, I was only a skype away to answer them, mostly in real-time. The process would have been slightly less effecient if we had time and potentiall language barries to deal with. We could potentially off-shore this later if needed, but in doing so I think you have to factor ‘communication costs.’ (mostly time delays) into the mix when calculating if this actually saves you anything in the long run.
I agree 100%. After working in an agile environment for close to 2 years now, I have become even more convinced that real-time collaboration is the glue that hold everything together. That simply can’t be done with off-shoring.
The first poster says ‘keep the requirements handy and keep them up to date’. That’s an agile antipattern. We talk to each other and work together to get the work done. We don’t spend out time documenting requirements and keeping them up to date.
Off shoring may appeal to the person looking only at numbers, but the true cost will be seen an increase in bugs, interruptions to the business by these bugs, more frequent releases to fix bugs, less time available to focus on new functionality, and a less than optimal agile team.
Hi Darrin, nice post. Offshore testing is fine as long as no one onshore needs to know if it works. If you want to produce quality you need to take some responsibility for it. If you want to pass the buck why stop in Asia? Let your imaginary friend do it 
I completely agree, the need to be sharing space and ideas as they occur and handle situations real time as they arise is in my experience critical. I’ve had some limited success with off-shoring well architected automated tests where the tests were designed on-shore and executed off-shore with results reporting to on-shore teams by 8:00 a.m. There was some value gained in running these tests as “smoke tests” to know the sanity of the build prior to full regression and testing. However, my gut belief and experience teaches me that being together expedites the communication process so well that testing is done far more often lessening the need for waiting for off-shore tests to render results..
Nice article Darrin!