Pankaj Nakhat's blog : All About Agile Testing

https://github.com/pnakhat/jbehave-selenium/tree/master/jbehave-selenium

I have created this project  to give some starting point for people who want to use Jbehave , Selenium with Page objects. Please clone this github repository.

I have been practicing Acceptance Test Driven development for years and since last few years I have been using BDD . BDD is a collaborative process to engage with business at the same time offering other benefits in Automation testing. There is no doubt BDD has changed the way we collaborate in an Agile team but also enhanced communication among stakeholders.

Some of the challenges I have seen while working on various projects, which I learnt from on my journey to do Behaviour Driven Development.

Process challenges

1. How to get business engage with this Idea of doing Testing  ?- I am sure this is easier said then done. How do I get my business stakeholder to engage in this process ? This scenario is like a sales scenario where you want to sell a great new product, but the problem is features are not immediately obvious.  Yes its about selling,  before its benefits becomes obvious.   Good thing about is BDD is natural language of expression, so getting involved is easier. BDD Scenarios makes so much sense in terms of a requirement. You can drive a product by an example in your own language, which is great. Its a great comfort when the Gap between requirement to development is negligible. Technical and Non technical people are speaking same language and you can validate your product in the very same language .
2. How to be consistent with the language  used in BDD ?  The whole idea of this process is about using a language which all stakeholders can understand and make sense from ? But one of the smells of BDD is using multiple style to define your behaviors. And result is simple, people are not consistent with their language of communication, so further consequence is mis communication and chaos.  Reason is simple “a” thing in my behaviors can mean few different thing. Other aspect of the same problem is different level of abstracted language people use, should I define my behaviors at 10000 ft or should it be more granular ? Well this depend  on the context, I don’t mind any approach where the language speaks the language of the problem we are trying to solve and its consistent.
3. Who should write the BDD ? I think it doesn’t matter who writes the BDD Scenarios. As long as different stakeholder participate in the process, and it is used as paradigm of communication it does not matter who writes it eventually. I have heard people getting in to discussion whether QA/BA/SME should write the acceptance tests. I think idea of doing BDD is to share the responsibility and getting everyone engaged in this process.
4. How to maintain my Acceptance tests going forward ?  I think the most important challenge in doing Acceptance Testing using BDD is, how does change in feature/code/data be managed. How the change should be managed in the acceptance tests? How not to fall in big trap of technical debt when big re factoring comes along and acceptance tests are well forgotten ? And I think most teams falls apart on these challenges more often then not. I have seen developers changing the acceptance tests without liaising with BAs and QAs when pressure of delivering eventually mounts. Similarly I have seen QA/BAs not managing to change the acceptance tests to reflect the change in feature. So basically challenge is to follow the same process when things are tight. Every change in feature should be managed as if its a new feature and should have a respective acceptance tests, agreed together with different SMEs.  Individual people on team should refrain making changes to acceptance tests, event though it may look trivial.Sometime it is difficult to trace acceptance tests against requirements, as in there is no obvious tooling in place to support trace the requirements against the tests. And if you are working on a large project it can be a nightmare to do so.  In this case having some means of tracing mechanism in place definitely helps. It can be as simple as a spreadsheet or usage of story maps if you are using tools like jbehave.

A Tester’s perspective on agile projects

Agile is no longer a buzzword or an unknown territory in the industry. Agile has come forward leaps and bounds since last few years and has seen a great transition from a newbie to a widely accepted methodology.

Testing on agile project was a paradigm shift for the traditional testing roles.  It needed a change in tester’s attitude from a laid back relay race oriented approach to an upfront involved role.  No longer had QAs needed to get something over the line to know the quality, as approach was rather to do the things right first time. But it’s easier said than done, how does it happen in reality? Does it actually happen?

Agile is a philosophy which believes in delivering constant value to customers incrementally and frequently, which is based on simple values of communication and feedback.  These two ingredients are the ingredients of a successful agile recipe.

A tester has an important role to play in agile projects throughout different phase of iterations. Iteration is generally a two week long activity where small requirements are analysed, coded and tested.

What QAs needs to do? I am outlining few of the basic QA activities in agile projects. Let’s simply break the QA activities in three phases in agile projects. However, this is no golden rule and it can be flexible according to the project situation. Agile QAs role is not limited to set of pre defined processes, as the methodology they are expected to play an agile role as per the situation.

Pre iteration : This is the time where requirements are analysed in detailed by the BAs and acceptance criteria are detailed out for that story. As QAs are immediate consumers of those requirements, so it is really important to verify these requirements early and often.

Story verification (Requirement Verification) : Agile testing is all about giving feedback early, not necessarily only by testing the requirements, but doing requirements testing early on. What should one be focusing on while doing requirements testing?

Looking at requirements for Clarity and Testability  - QAs really need to look at the requirement / stories upfront to figure out that requirement are unambiguous and testable.

Unambiguous- I think unambiguous in agile has little different context compared to typical waterfall project.

- Requirement should be small enough to make sense in the context.

- Acceptance criteria (Stories are generally broken in to acceptance criteria) should not be duplicate or overlapping from different stories.

However, doing that can be really difficult and can only be achieved with really good communication between Dev/BA/QA.

Testable: Testability aspect of the story requires QA to scan through the story to see what needs to be done to test the story. These factors can generally be.

- Finding hidden requirements

- Environment

- Test data

- Dependency on other requirement.

Getting these details early helps the story to be prioritised accordingly in the backlog, and allows smooth execution of the story in the iteration.

QAs also do participate in the iteration planning meeting to help estimating and giving testing perspective to the team to come up with developer estimate. QAs have big role to play in the iteration planning, as some of the implicit requirements gets escalated by

QAs.

a) QA Activities In  The Iteration

Once QAs are happy with the acceptance criteria of the story they can work towards defining the acceptance tests for the story. Acceptance tests are requirements in terms of a test, so that it can be executed before hand to know what is expected out of requirements. These acceptance tests are generally automated and used by developers to drive development,  Acceptance tests should not be covering too many corner case scenarios as this would create unnecessary delay in getting the story done by the developers and may end up in producing too many similar automated tests.

People often fail to understand that acceptance testing in agile projects is different from traditional projects, as in agile projects it happens before the software is delivered unlike traditional project where it happens at end of the software lifecycle. In agile projects acceptance tests are generally automated so that it can be run as regression tests.

Automation testing is really important for any agile project as frequent builds requires a short feedback cycle hence regression testing needs to be quick and accurate. Automation testing is different from traditional automation testing. In agile projects automation testing is practised by all levels be it Developers, QAs or BAs. It does not necessarily mean that everyone needs to be writing the test code. Involvement from everyone on automation testing increases the relevance of the tests and often helps in finding and implementing the right tests.

It’s always been debatable as who owns the automation tests in agile projects? And for me it’s more of a responsibility then a role.  In my experience most effective automation tests are achieved when Dev/QAs work together on it.

Use Automation to the advantage not for sake of doing it

Automation in agile is quiet controversial, as people try to automated everything and end up in a trap of having a long feedback cycle. Automation is meant for giving early feedback on the latest code, and automation should be limited to what is worth Automating and what is not. Every automation test written has a cost against it. The cost can be seen as

Cost of continuously running it (Increased feedback cycle). This cost of automation should be compared against cost of not running it. So, a question needs to be asked what if a test is not automated. What we going to lose, what would be the cost of fixing the stuff around the code for which we are losing the coverage? Is it cheaper to test manually?

And it might not be straight forward finding the worth of a test. It’s a contextual decision and also depends on size of the project and number of people involved in it. Simply putting this in other words mean Longer feedback cycle = More people losing time in getting instant feedback.

The typical QA activities in the iteration are to continuously measure the Quality of the software. QAs participate in the story handover to the developers to make them understand the testing requirements of the story, so that developers are enabled for Test Driven Development (TDD) for their code, which is a backbone for agile development practices. Also handing over the acceptance tests and making developers understand about the testability aspect of the story often catches the most common defects even before they are delivered to QAs.  These activities require a high level of communication with the developer and BAs to clarify things on the fly and making sure we are building the product right first time. QAs may also have to pair with developers work together on the story or tests for the story to get the better understanding of the requirements.

QAs try to resolve as much issue before hand by actively participating in the overall process.  It also becomes imperative that once the story is delivered, it is tested properly with all the aspects and ideally on a proper environment.  Once the QAs are happy with the stories/requirements, they sign off that piece of work in a typical manufacturing way and hand it over for further process in the line which generally a showcase to the business.

As a QA it is also important to think beyond the written requirements and to have a desire to break attitude, and that’s where QAs do exploratory testing to execute ‘out of the box’ scenarios and also do negative testing to make sure software is really robust. Exploratory testing is a not at all about executing a pre defined testing scenarios, it is more of exploring of software beyond test cases but at the same time keeping the focus around some specific requirements.

I have tried covering most of the aspects of the basic QA activities of a QA in an agile project, however it is not limited to these activities and QAs should play more collaborative role as and when needed to be able to provide more visible and transparent feedback.

Download the article

We have upgraded ourselve to the new Twist Beta.

And this comes with some cool features.

- Inline Reporting

- Proxy settings

I have already spoken about this in my previous posts.

- Automatic variable assignment to an UI object while recording.

The way we have Incorporated the object repository concept in Twist by creating a single Java file to hold all the object information.

Rather then using object definitions directly in the script, which makes them brittle and hard to manage, we use the objects from the repository and if something changes about that object, we know where exactly to find that object and also that avoids making thousand changes if that particular object is being used quiet a lot.

And last not the list recorder seems to be improved in the new Beta version and its really stable so far.

I have worked on various tools for web automation so far, and I see web automation as a simple mechanism if we break them in to generic actions (Be it any tool/Language).

I would call them Four basic and only pillars.

1) Click – Button, Link, Image
2) Select Drop down, option, radio etc
3) Navigate URL
4) Verify Text (On a page, in a Textbox, in a Table, in a Div , page title)

Objects are the most difficult things to manage in any automation tool/framework.

Though QTP gives a good way of managing objects via Object Repository manager, Still a few tidy things could make your scripts more robust and effective, from OR’s perspective.

1) Browser Objects :

If Title is same for all the pages in the browser, through out the application, only one browser objects should be used to have less overheads. By saying ‘One Browser’ object, I mean that all the browser objects in all the repositories should have the same name. This would help us more in writing and maintaining scripts, as you need not worry about n different names for the browsers at different places.

User ‘Title’ property as single identifier, this would help in managing conflicts well.

Get rid of the prperties (Remove from the description properties ) like

1) ‘openurl‘, as this makes the object dependent on the URL and even a slightest of the change in URL would make the script break.

2) ‘version’ , ‘application version’: This will make the scripts independent of Browser type and Versions.

2) Page Objects




- Don’t use URL property in the object description properties, wherever possible

- If all the pages in the application have same title use the ‘title’ property to identify the page in all the object repository. Make sure you keep the name same for the all the pages.

This would reduce the overhead of using different object name in the scripts and maintenance would be very easy.

People may disagree here, as keeping the name same for all the pages, does not help in terms of documentation or rather helping in identify which page are we referring to from different places?

My Answer is : In a good framework, people work on multiple well defined functions, so function reference is good enough to tell me who is doing what?

Also, it is a good practice to have small manageable repository (.tsr, .bdb) file in QTP for good performance and easy maintenance.

So, one can have a repository for each script (Functionality), which makes it obvious that even if I am using a single name for all the pages, it becomes quiet evident by looking at repository that which Repositor is pointing to which particular page/pages.

Another reason is, keeping a single name for browser and page would mean a single heirarchy, irrespective of number of Object Repositories.

So, even if ten repositories are shared in a script/Test, you don’t have to worry about thousand different heirarchies.

However, the next question is, when you see all your objects from different OR’s under one heirrachy, isn’t it confusing, in terms of identifying the object?

Answer is YES it can be confusing, if objects are not named properly.

E.g? Button1 should be rather named as Button1_loginpage, this would help me compensating for not using different pages at the same time taking benefits of using single heirarchy.

How Does a Single Heirarchy helps in scripting?