Pankaj Nakhat's blog : All About Agile Testing

Software testing has changed dramatically since the inception of Agile and Scrum. These methodologies are now widely accepted and without any doubt offers better way of delivering software. However Agile and Scrum also highlighted some of the challenges in the testing processes which possessed greater risk of failure , if not thought about in the over all process.

Some of these challenges becomes inevitable , if some of the critical processes in scrum like story writing, estimation and TDD falls apart.

Some of the high level Testing challenges in Agile and Scrum

And I think these are some of the core points if taken care can make huge impact on the overall testing process and for that matter in delivery.

Having the right Requirement

I see this as a biggest pain points and root of other problems. Having a good story can change how the requirement is delivered and tested. I refer this from the book “User stories Applied”, that a story should inhibit following features INVEST

I – INVEST
N – NEGOTIABLE
V – VALUABLE
E – ESITMATABLE
S – SMALL
T – TESTABLE

Each of these characteristics are self explainable and however trivial they may sound, they have huge impact on the quality of the requirement delivered in the end.

For instance a large story which can pretty much eat good few sprints is more likely to introduce bugs, as it will not be ready (DONE) by end of the sprint for review. And also large stories create confusion for developer as they get out of context very soon, similarly its nightmare for testing as well since focusing on a big chunk of feature creates opportunity for bug and overlooking.

Also, a badly written story can lead a wrong estimation which can cause capacity issues for testing as chances are some stuff will have to rolled over to next sprint or rushed through. Not only that a badly estimated story can lead to greater risks in further sprints, as it will be eat up some of the valuable time needed to prepare for next sprint’s requirement.

I strongly believe that testers in Agile project can add lot of value by looking at requirements nice early and reviewing it from a tester’s point of view. Some of the questions a tester need to asks are

- Does this acceptance criteria makes sense?
- How can I test it? (What, where and When )
- Is it not a duplication of work we did earlier ?
- Is this story too small or big ?

Automate Automate Automate

Automation will help maintaining a tight grip on the changes. Automation should be done at all levels and with involvement of QAs). Why do I say that? As Agile accepts changes quiet frequently and focuses on delivering features every sprint. it is obviously you would not get much time to plan, write and manage those tests, hence it is imperative that test automation is thought about while writing the code. And QAs should play important role in discussing, planning and thinking about those tests upfront, and discuss them with BAs and Developers. Build as much input as possible and then take right decisions as in which tests to be automate as unit, integration or UI tests.

Automation are assets and needs to be maintained, re factored and looked after like production code. I have seen automation tests becoming big bottleneck to maintain and trace and these problems can be solved by doing some simple things regularly .

- Make sure your automated tests are traceable against requirements (There are tools available to do that , if not use spreadsheet ). This will make sure you can track the tests against a requirement and if needed re factor, extend or retire it.
- Automated at all levels – UI tests are expensive and not conducive to tests each and everything. Good unit and integration test helps in building better coverage and helps in giving faster feedback. End to End tests are important and should be exclusive and focus on workflow and driving application end to end.
- Get testers involved in writing unit and integration tests, they will provide valuable feedback.

Defining the DONE Criteria

I think this is the first and foremost process to be in place from Day1.

What is DONE Criteria ?

DONE criteria are high level check list items which needs to be satisfied in order to claim a story is DONE. DONE Criteria should have technical, process, functional, non-functional, configuration aspects of the software. DONE criteria forces teams to take holistic view on the stories rather then code completing the feature.

Some of the common aspects of done criteria

- All acceptance criteria met. (Functional)
- Performance not degraded. (Non Functional)
- Code coverage achieved and all acceptance tests automation. (Technical )
- All build passed and application deployed on UAT/Staging environment. (Environment/Configuration)
- Exploratory testing performed and QA signed off.
- Demoed to stake holders on proper environment. (Process)
- All known bugs closed and tested (Process)

By doing this teams will make sure that software produced is of great quality and all aspects of the applications are taken care. Doing this repeatedly will make sure testers are not doing catchup but they are part of the story and sign off process. Software produced in the end can be released and should be production ready.

Exploratory Testing

This is the most misunderstood and ignored aspect of Agile Testing. Exploratory Testing advocated testes/BAs to test a feature by exploring it beyond the test scripts. At the same time it is not about clicking around the application without proper context.

Exploratory testing should be done every sprint on set of features on a proper environment. Testers should use all their skills to think out of the box and try to fearlessly test the system within the context of the features, however not limited to that feature.

Success of Agile Testing depends on working collaboratively between different stakeholders of the project and learning and rectifying quickly.