Pankaj Nakhat's blog : All About Agile Testing

https://github.com/pnakhat/jbehave-selenium/tree/master/jbehave-selenium

I have created this project  to give some starting point for people who want to use Jbehave , Selenium with Page objects. Please clone this github repository.

I have been practicing Acceptance Test Driven development for years and since last few years I have been using BDD . BDD is a collaborative process to engage with business at the same time offering other benefits in Automation testing. There is no doubt BDD has changed the way we collaborate in an Agile team but also enhanced communication among stakeholders.

Some of the challenges I have seen while working on various projects, which I learnt from on my journey to do Behaviour Driven Development.

Process challenges

1. How to get business engage with this Idea of doing Testing  ?- I am sure this is easier said then done. How do I get my business stakeholder to engage in this process ? This scenario is like a sales scenario where you want to sell a great new product, but the problem is features are not immediately obvious.  Yes its about selling,  before its benefits becomes obvious.   Good thing about is BDD is natural language of expression, so getting involved is easier. BDD Scenarios makes so much sense in terms of a requirement. You can drive a product by an example in your own language, which is great. Its a great comfort when the Gap between requirement to development is negligible. Technical and Non technical people are speaking same language and you can validate your product in the very same language .
2. How to be consistent with the language  used in BDD ?  The whole idea of this process is about using a language which all stakeholders can understand and make sense from ? But one of the smells of BDD is using multiple style to define your behaviors. And result is simple, people are not consistent with their language of communication, so further consequence is mis communication and chaos.  Reason is simple “a” thing in my behaviors can mean few different thing. Other aspect of the same problem is different level of abstracted language people use, should I define my behaviors at 10000 ft or should it be more granular ? Well this depend  on the context, I don’t mind any approach where the language speaks the language of the problem we are trying to solve and its consistent.
3. Who should write the BDD ? I think it doesn’t matter who writes the BDD Scenarios. As long as different stakeholder participate in the process, and it is used as paradigm of communication it does not matter who writes it eventually. I have heard people getting in to discussion whether QA/BA/SME should write the acceptance tests. I think idea of doing BDD is to share the responsibility and getting everyone engaged in this process.
4. How to maintain my Acceptance tests going forward ?  I think the most important challenge in doing Acceptance Testing using BDD is, how does change in feature/code/data be managed. How the change should be managed in the acceptance tests? How not to fall in big trap of technical debt when big re factoring comes along and acceptance tests are well forgotten ? And I think most teams falls apart on these challenges more often then not. I have seen developers changing the acceptance tests without liaising with BAs and QAs when pressure of delivering eventually mounts. Similarly I have seen QA/BAs not managing to change the acceptance tests to reflect the change in feature. So basically challenge is to follow the same process when things are tight. Every change in feature should be managed as if its a new feature and should have a respective acceptance tests, agreed together with different SMEs.  Individual people on team should refrain making changes to acceptance tests, event though it may look trivial.Sometime it is difficult to trace acceptance tests against requirements, as in there is no obvious tooling in place to support trace the requirements against the tests. And if you are working on a large project it can be a nightmare to do so.  In this case having some means of tracing mechanism in place definitely helps. It can be as simple as a spreadsheet or usage of story maps if you are using tools like jbehave.

I have been wondering for a while, if there was a way to get all my BDD steps in one place to so that I can do a search on the existing steps. As the Library of steps grows gradually often finding existing steps becomes a lazy and tedious activity. And then we end up with multiple steps doing same things.

Jbehave reports can be ebabled to show something which is called Storynavigator. Its a client side javascript library which enabled browser to read meta data generated by cross referencing in Jbehave. Cross referencing can be enabled very easily from the configuration and it then generated the meta data which can be used by this story navigator.

How to enable cross referencing in Jbehave configuration?

 @Override
    public Configuration configuration() {
    	CrossReference xref = new CrossReference();
 
        Class embeddableClass = this.getClass();
        // Start from default ParameterConverters instance
        ParameterConverters parameterConverters = new ParameterConverters();
        // factory to allow parameter conversion and loading from external resources (used by StoryParser too)
        ExamplesTableFactory examplesTableFactory = new ExamplesTableFactory(new LocalizedKeywords(), new LoadFromClasspath(embeddableClass), parameterConverters);
        // add custom coverters
        parameterConverters.addConverters(new DateConverter(new SimpleDateFormat("yyyy-MM-dd")),
                new ExamplesTableConverter(examplesTableFactory));
        parameterConverters.addConverters(new MyConverter());
        parameterConverters.addConverters(new StudentConverter(new ExamplesTableFactory()));
 
		return new MostUsefulConfiguration()
            .useStoryLoader(new LoadFromClasspath(embeddableClass))
            .useStoryParser(new RegexStoryParser(examplesTableFactory)) 
            .useStoryReporterBuilder(new StoryReporterBuilder()
                .withCodeLocation(CodeLocations.codeLocationFromClass(embeddableClass))
                .withDefaultFormats()
                .withFormats(CONSOLE, TXT, HTML, XML)
                .withCrossReference(xref))
            .useParameterConverters(parameterConverters)    
            .useStepMonitor(xref.getStepMonitor());
    }

Now the next step is to copy the JS libraries in your project and make sure they are copied in the view directory. Download the all the files and folder from here https://github.com/jbehave/jbehave-core/tree/master/jbehave-navigator/src/main/resources. Copy them under src/main/storynavigator folder in your source.

Add this confif in your POM files, so that these files are copied in the Jbehave view directory

        ${basedir}/src/main/storynavigator
        ${project.build.directory}/jbehave/view
        false

Now every run will generate a file called navigator.html inside /target/jbehave/view.

First tab is Stories, it will allow searching on story description, narrative and meta tag. Second tab is the Steps Tab, this will allow searching all the steps in your step classes.

Bit of learning – You will not see data in Stories tab for you story if no meta tags are used in the report.

If you are using Jbehave and wonder how to execute your scenarios with different set of data without repeating the steps, so that your scenarios don’t blow up repeating same steps with different set of data.

You can also refer to this scenario as data driven testing, as each of the row in your data table becomes one test case.

In the example below lets say I am testing a calculator sum function, and for that I need to have several test cases to test with different data. So basically steps looks like this, however we need to give different input and assert output.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

 
 
Scenario: As as user I want to test calculator sum function 
Meta:
@author pankaj
 
Given calculator takes <input1> and <input2>
Then the sum is <sum>
 
Examples:
|input1|input2|sum|
|2|3|5|
|4|5|9|
|10|12|22|

And steps implementation for the above written steps will need to use Named parameter. So basically Named parameter (Which is usually in <>), will be replaced by the data in the table below the steps in the story. In the above example there are three named parameter in my story.

In the steps implementation, you step needs to explicitly use the named parameter in the method. Using named parameter in the steps tells jbehave that it needs to replace this with the data in the table specified below/or in another file.

Steps class would look like

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

package org.qainfolabs.jbehave.steps;
 
import java.math.BigDecimal;
 
import junit.framework.Assert;
 
import org.jbehave.core.annotations.Given;
import org.jbehave.core.annotations.Named;
import org.jbehave.core.annotations.Then;
import org.qainfolabs.app.Calculator;
 
public class CalulatorSteps {
 
	private BigDecimal input1;
	private BigDecimal input2;
 
	@Given("calculator takes <input1> and <input2>")
	public void readFruits(@Named("input1") BigDecimal input1,
			@Named("input2") BigDecimal input2) {
		this.input1 = input1;
		this.input2 = input2;
	}
 
	@Then("the sum is <sum>")
	public void doSum(@Named("sum") BigDecimal sum) {
		Calculator calc = new Calculator();
		Assert.assertEquals(sum, calc.sum(input1, input2));
	}
}

After the scenario is executed Jbehave will produce a detailed report of the scenario, showing the steps repeating with replaced parameter with each data set.