Archive for April, 2012

Cucumber And SpecFlow Aren’t Test Tools

…. although it’s certainly possible to use them that way. But why would one such a thing, de-tuning what is really a rather nice way of getting product owners and developers to talk to each other?

At least that’s what I think at the current state of my exploration of the world of Given-When-Then.

It comes down to trust and boredom:

Feature: Our new application

Scenario: Users use the system
Given that we want this application to work well
And that there are requirements
When you build it
Then it should meet the requirements
And delight users.

This is the ultimate trust: The spec is short and sweet. It lacks specifics, but there is hope that the CEO will not get bored when reading it: The buck stops with her if requirements are not met and users are not delighted. Technical detail for which she has no time is omitted. Actionable success criteria are at least outlined (“Ask Larry if we’re delighting users yet. Have Malcolm’s folks see if we are meeting requirements.”)

Unfortunately, it’s not very helpful. So let’s look at paranoia instead:

Feature: Create Bus Stop
    We need to be able to enter bus stops into the
    system     so we can have busses arriving at them. 
    There must be no duplicate bus stop numbers and 
    no two stops must be in the same place.

Scenario: Bus stop creation succeeds
    Given there are the following existing bus stops:

        | Bus Stop     | Latitude   | Longitude   |
        | 12111 | 49 20' 0'' | 122 1' 23'' |

    And I enter bus stop no. 44444, lat. 50 1' 12'', long. 122 23' 02'' into the 'Create Stop' form
    When I click 'Create'
    Then the stop with no. 44444, lat. 50 1' 12'', long. 122 23' 02'' is in the list of bus stops. 

Scenario: Cannot create a stop if the stop number already exists
    Given there are the following existing bus stops:

        | Bus Stop     | Latitude   | Longitude   |
        | 12111 | 49 20' 0'' | 122 1' 23'' |

    And I enter bus stop no. 12111, lat. 50 1' 12'', long. 122 23' 02'' into the 'Create Stop' form
    When I click 'Create'
    Then the error message "A stop with this number already exists." appears.
    And the stop with no. 12111, lat. 50 1' 12'', long. 122 23' 02'' is not in the list of bus stops.

Scenario: Cannot create a stop if the stop is in the same location as an existing stop
    Given there are the following existing bus stops:

        | Bus Stop     | Latitude   | Longitude   |
        | 12111 | 49 20' 0'' | 122 1' 23'' |

    And I enter bus stop no. 10000, lat. 49 20' 0'', long. 122 1' 23'' into the 'Create Stop' form
    When I click 'Create'
    Then the error message "A stop already exists in this location." appears.
    And the stop with no. 12111, lat. 50 1' 12'', long. 122 23' 02'' is not in the list of bus stops.

This is great for testers:

  • It’s easy to add new tests which re-use existing step definitions. For example, setting up test data for new scenarios with “Given there are the following bus stops” is straightforward.
  • Lots of parameters are available to write edge test cases. What happens if I enter bus stop no.  –9999999999999999999, lat.  5000 –12’ 0;;, long. 0 0’ 0’’? We can find out – plenty of opportunity to add scenarios without the need for a whole lot of step definition coding.

Unfortunately, it’s boring and obscure:

  • Too much detail. Who has the time? Come to the point.
  • How can I tell how this fits into the big picture? Are we missing something?
  • It’s repetitive and makes your eyes glaze over.
  • Ze Inglisch kann be bat.  “Given there are the following existing bus stops”, followed by a table of only one?

Alternatively, one might tell a story:

Feature: Bus stop creation
    We need to be able to enter bus stops into the
    system so we can have busses arriving at them. 
    There must be no duplicate bus stop numbers and 
    no two stops must be in the same place.

Scenario: Successful bus stop creation
    Given that I enter a bus stop number and location
    And the bus stop number doesn't exist yet
    And there is no other bus stop in the same location
    When I try to create the bus stop
    Then the bus stop should appear in the list of stops.

Scenario: Bus stop number already exists
    Given that I enter a bus stop number and location
    But the bus stop number already exists
    When I try to create the bus stop
    Then an error message appears
    And the bus stop is not created.

Scenario: A Bus stop already exists in this location
    Given that I enter a bus stop number and location
    But there is already a bus stop in this location
    When I try to create the bus stop
    Then an error message appears
    And the bus stop is not created.

Testers won’t be happy:

  • Not possible to write comprehensive tests without new step definition code.
  • Too much ambiguity. What exactly is a “location”?

But it’s less boring:

  • It’s possible to read it like a story about what the system can do and what rules it has, without excessive repetition which obscures the big picture requirements and business value.

So what is the point of the story if it doesn’t give us decent tests? From “The Cucumber Book”:

“Cucumber might just seem like a testing tool, but at its heart it’s really a collaboration tool.”

For example:

“Looks like we’ve got the the requirement of no two bus stops  in the same location covered. I just thought of it – what if somebody enters a stop which is very close to an existing one?”

“Hmm. Let me have a look how we implemented that. Here it is – We check for an exact match in latitude and longitude:”

[Given(@"there is already a bus stop in this location")]
public void GivenThereIsAlreadyABusStopInThisLocation()
{
    CreateBusStopInSameLocationAs(BusStopDataEnteredByUser());            
}

private void CreateBusStopInSameLocationAs(BusStop busStop)
{
    const string stopNumberWhichDoesntExistYet = "99999";
    var existingStop = new BusStop()
        {
            StopNumber = stopNumberWhichDoesntExistYet,
            LatDegrees = busStop.LatDegrees,
            LatMinutes = busStop.LatMinutes,
            LongDegrees = busStop.LongDegrees,
            LongMinutes = busStop.LongMinutes
        };
    _busStopController.Create(existingStop);
}

“Could we change that so that  we can’t have a second bus stop within 30 meters of an existing one?”

“OK…”

Arguably, this conversation might not have happened without the story. We are collaborating more because the story is told in a way that’s accessible to all participants:

The Given-When-Then story is executable documentation, accessible to non-technical stakeholders as a means of seeing what the application can and cannot do.

The tests exercise the code, telling the  “how we do it” part of the story: Click this button, fill in that field, call the method, …

The Step Definitions (… or fixtures, etc.) could be seen as a way of telling how developers interpreted the story. That’s the part I could never figure out before: They aren’t a specification. They aren’t tests. They sort of glue the specification and the tests together, but the usual reason why they need to change seems odd: They need to be modified because either the story or the underlying tests changed. Why did they change? Presumably it was because:

  • Product owners changed the story of what the system can do in the Given-When-Then specification.
  • Developers changed the way they interpreted the specification: They renamed classes/methods/variables, they changed method signatures or behaviours, etc.

In other words, the domain language evolved. New concepts were added to the ubiquitous language, existing ones were better understood or changed.

From a collaboration standpoint, “step definitions” could be seen as an interpretation layer, a  “We Thought You Meant…”

Given readable code, consistent levels of abstraction and good naming developers and product owners might look at this and get a better understanding of  what is being built.

The value compared to having requirement docs in Word and just doing TDD style tests comes from relating requirements to implementation in a way which is accessible to business stakeholders, encourages conversation and is executable at the same time:

  • Given-when-then (What)
  • Step definitions  (“… Thought you meant…”)
  • Tests (How)

Anyway, a story about stories. Given less boredom and more trust when we build the app then we’ll get better software?

Advertisements

April 2, 2012 at 6:31 am 2 comments


Twitter

   @robertreppel