Pankaj Nakhat's blog : All About Agile Testing

If you are using Jbehave and wonder how to execute your scenarios with different set of data without repeating the steps, so that your scenarios don’t blow up repeating same steps with different set of data.

You can also refer to this scenario as data driven testing, as each of the row in your data table becomes one test case.

In the example below lets say I am testing a calculator sum function, and for that I need to have several test cases to test with different data. So basically steps looks like this, however we need to give different input and assert output.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

 
 
Scenario: As as user I want to test calculator sum function 
Meta:
@author pankaj
 
Given calculator takes <input1> and <input2>
Then the sum is <sum>
 
Examples:
|input1|input2|sum|
|2|3|5|
|4|5|9|
|10|12|22|

And steps implementation for the above written steps will need to use Named parameter. So basically Named parameter (Which is usually in <>), will be replaced by the data in the table below the steps in the story. In the above example there are three named parameter in my story.

In the steps implementation, you step needs to explicitly use the named parameter in the method. Using named parameter in the steps tells jbehave that it needs to replace this with the data in the table specified below/or in another file.

Steps class would look like

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

package org.qainfolabs.jbehave.steps;
 
import java.math.BigDecimal;
 
import junit.framework.Assert;
 
import org.jbehave.core.annotations.Given;
import org.jbehave.core.annotations.Named;
import org.jbehave.core.annotations.Then;
import org.qainfolabs.app.Calculator;
 
public class CalulatorSteps {
 
	private BigDecimal input1;
	private BigDecimal input2;
 
	@Given("calculator takes <input1> and <input2>")
	public void readFruits(@Named("input1") BigDecimal input1,
			@Named("input2") BigDecimal input2) {
		this.input1 = input1;
		this.input2 = input2;
	}
 
	@Then("the sum is <sum>")
	public void doSum(@Named("sum") BigDecimal sum) {
		Calculator calc = new Calculator();
		Assert.assertEquals(sum, calc.sum(input1, input2));
	}
}

After the scenario is executed Jbehave will produce a detailed report of the scenario, showing the steps repeating with replaced parameter with each data set.

Testers on agile projects have to often deal with multiple tasks in an iteraton.

Before Sprint

-          Working closely with BA on upcoming stories

-          Where ever possible produce acceptance criteria / test of the stories to be done in coming sprint

-          Understanding review of the stories

-          Challenging the ambiguous requirements

-          Challenging big stories

In the sprint

-          Work with development team on implantation of the acceptance tests of the stories.

-          Pair with developers if it helps the team to get stories moving quickly and reducing the feedback time

-          Discuss different levels of testing to be done for the stories . Don’t think you don’t have any responsibilities towards unit and integration tests.

-          Have an understating of implementation of the tests being written and challenge a implementation wherever you think the code will not be exercised in the right way.

-          Make sure the feedback is fast and delivered stories are tested on proper environments and meets the Done criteria.

-          Make sure you builds are green and tests are passing all the time

-          Work with business / BA on the delivered stories to sign off it.

-          Document and communicate the bugs according to your team process.

-          Implement more comprehensive automated tests if needed.

-          Make sure Non functional requirements are met

Post sprint

-          Work with business/ BAs on the demo of the story, as I think QAs develop deep understanding of the requirements.

-          More exploratory testing.

-          Look at performance/load/stress criteria of your application every now and then.

-          Close the fixed bugs and make sure proper regression tests are added.

This list is not a comprehensive tasks lists and it is meant only for guideline purposes.

WebDriver is really  a good clean and powerfull API to test the browsers. Obviously now Selenium 2.0 is backed by WebDriver and there are different options you can use webdriver in different way. (With and Without a Proxy server) But originally WebDriver came with an idea of talking to browsers natively to driver then to achieve faster and better response. How is it different from Selenium. Please see the post . So, I would focus on this post to write a simple Google search test using webdriver.

I am assuming you have got a basic Java project setup where you can write tests in Java, and also you can choose any unit testing framework of your choice. WebDriver is a generic interface which then gets implemented by different available drivers (browsers) to achieve the consistency in the API. However, there are still some difference between the drivers, but we will ignore that for this posts, as basic API is consistent across the driver. Having an interface independent of driver implementation helps to write driver independent  tests, so we can write a test in WebDriver and run that against any available driver (Firefox, Chrome, IE and Iphone driver).

I must admit that chrome driver is little immature at the moment, but I am sure it should be good soon. For my example I will be using Eclipse as the IDE and TestNG as the unit testing framework.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

package org.wd.tests;
 
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.Test;
import org.wd.annotation.DataDriven;
 
public class TestGoogle{
 
	/* This creates a field of type WebDriver Interface*/
	protected WebDriver driver;
 
	@Test
	public void testGoogle(){
		/*This will crate a new Instance of Firefox driver*/
		this.driver = new FirefoxDriver();	
		/*This will launch www.google.co.uk on the browser*/
		driver.get("http://www.google.co.uk");	
 
		/*Identify the elements to interact with */		
		WebElement searchTextBox = driver.findElement(By.name("q"));
		WebElement searchButton = driver.findElement(By.name("btnG"));
 
		/*Take actions now - Set the text field*/
		searchTextBox.sendKeys("Pankaj Nakhat");
		/*Click on the searchButton*/
		searchButton.click();		
 
		/*Verify HTML response contains some text*/
		Assert.assertEquals(driver.getPageSource(), "Pankaj Nakhat");
		driver.quit();
	}
}

I have been involved in numerous interviews to hire Agile tester for different organizations and always wondered what should be the right skill set you should look for in a good agile tester.

Agile testing is not a rocket science and you still need people with good testing skills, that is a prerequisite.  But most of the organizations get confused between hiring a QA in terms of desired technical skills and QA Skills.

If you see most of the job specs floating around in the market, they have standard specs  viz Selenium, Fitnesse etc etc.  So, basically people seek technical tester to do test automation and hence they get glued to some particular job specs while hiring Agile Testers.  Where do you draw a line that a tester is technical enough to do the job, at the same time posses necessary Testing Skills?

Do people miss a trick here, in forming a good checklist for an Agile Tester?

A good Agile tester should be technically sound, there is no second thought about it, but then why can’t people hire good developers to do testing job in Agile projects.  That’s where I think QA specific skills should be looked for while hiring Agile Testers,  at the same time assessing their technical skills. I will get to the point of “QA Skills” in a while, but before that I want to touch upon on Technical skills of an Agile Tester.

Technical skills of an Agile Tester should not be only limited to knowing some specific tools, I think an Agile Tester should have a good programming aptitude. So this will enable them to pick up any new technology quiet quickly.  They should then apply their automation/tools experience with any given technology.

Also, an Agile Tester should be aware of the application architecture, hardware, tools and interaction between different systems. That enables them to think about the testing from much more wider perspective.

What QA skills one must look for whilst hiring Agile Testers?

1)      Good analytical skills to think about the requirement from users/testers perspective to verify a requirement and also testing it on adverse scenarios.

2)      Ability to communicate with business to challenge and asses a requirement.

3)      Proactive: An agile environment needs testers to be proactive to foresee any issues that may block the team in anyways.

4)      Ability to communicate well with developers:  I think this is equally important like interaction with business.   Testers play a role of a bridge between Business/BAs and Developers. And thats where they should not be technically deprived that one end of bridge become weaker.  I am not saying that you just need technical skill to be able to communicate with the developers.

5)      Good time/task management:  A Tester needs to play different roles in any  iterations/sprints and he should be able to priorities his work to be able to carry out the most important activities in the context of a sprint.

6)      Quick Learners : This is an attribute which is great to have in anyone, but I would asses a candidate on his learning ability.

I was looking at selenium Bromine out of interest, and could infer some quick feedback.

1) The workflow to create a Requirement To Test is too messy.

2) UI Is very flat, so cant make much difference between the tabs.

3) Functionality Just works ok, however you have to upload tests created from Selenium IDE to make it the test work, which I don’t think is Ideal.

4) Overall process of running a test is quiet complicated as you get confused between hierarchies of the test.

I was really excited about it, particularly with the new IDE changes, but it turned out to be a disappointment.

I will make it a quick one :

- IDE still does not support instantiating a VBScript class outside of the script file.
- Intellisense does not work with the variables.

I had a first look at Twist V1.0 and it looks impressive and stable.

Here are my findings.

1) After I installed, I tried installing the subclipse (After little bitof struggle, I figured out that you need to de select the ‘Revision graph module’ from the subclipse, and then you can move ahead successfully with the installation

2) As soon as I checked out my old Test suite ( created from Twist beta), it asked me to upgrade the project. It has a backup projecy feature now. But I didn’t do any backup and moved ahead with the upgrade, process was quick and smooth.

3) Executed one of the scenarios and worked smoothly. so far so great.

One quick noticeable change is speed of execution of tests and it looks much faster and smoother in V1.0

4) The Tag management featured is quiet good, now I can manage tags of multiple tests in one go.

5) Now it preserves the test execution history if tests are executed from the editor, which was really lacking in the previous versions. A Great feature to have.

Features I have not tried so far : Recording, Rules table and Context but I am sure it must be good.