Zombie Agile: Bad Things Done Right

June 19, 2010 at 10:51 am 4 comments

I currently have the pleasure of working in one of those “waterfall-ish” shops where things actually work: Profitable projects get delivered in fairly predictable timeframes by sane people. They have a fairly “traditional” setup with business analysis folks who write old-style requirements documents (… few if any new-fangled “user stories”…).  These are given to development for implementation, to be manually QA’d by the QA team.  A few lone rangers not withstanding, practically nobody seems to write any tests.

How can this be … ? Aren’t we supposed to TDD/BDD/scrummerize everything, practicing kanbanilism for good measure? Blasphemous whispers in the Church Of Agile; Petty doubts: Have I joined a fringe cult …. ?

So what’s the difference? How do they get away with it?  Situations where I’ve encountered “traditional” software development life cycles that actually worked all had the same combination of traits in common:

A healthy “that was then, this is now” attitude to requirements.  A recent project is a good example: The “traditional” business analyst–produced requirements remained a living document which was updated throughout the delivery of the project. It could thus be relied upon to closely match what was implemented in the code. As much as anything, the habit of keeping it up to date led to many more conversations between BAs and developers than would otherwise have taken place, greatly helping everybody to understand the problem domain.  Very importantly, the document also served as vital input for doing the second part of the trick:

Relentless QA. Unsurprisingly in the absence of automated test coverage, there were many bugs. QA used the “living” requirements document as their guide to making manual test plans. They were very well-organized in executing them and very proactive in sharing the results with the developers. The resulting back-and-forth further helped to grow a shared understanding of the system and it’s requirements.

Smooth Deployments. Calling it “continuous integration” would be a stretch, but they had their builds and deployments organized in a way which allowed for one or two QA releases per week.

Awareness. They were aware of test driven development, the potential advantages of Agile and many of the techniques. However, they had a large legacy code base and many organizational structures and developer habits which did not lend themselves to “Agile” without major changes and a lot of experimentation.  It was a matter of “Koennen vor Lachen” (… German for “Easier said than done”.  It doesn’t translate well.).  Yes, sounds great, but how do we start … ?

Hard to say. Trying to do “Agile” without test-driven development is an extremely hazardous idea.  At the same time doing TDD in legacy situations is a tall order. For a team of developers to start consistently writing tests in brownfield projects takes a lot of new ways of thinking and technical savvy which I feel absolutely has to go hand in hand with appropriate ways of managing the process: Without knowledgeable support and commitment by non-technical stakeholders the effort will die. (Very common scenario/attitude: “We don’t have time to write tests.”)

The practices of “waterfall done right” could serve as initial steps when iterating towards “real agile”. Compared to a seasoned SCRUM / Lean team at their best,  “Zombie Agile” with living requirements, relentless QA and smoothed-out deployments is expensive, inefficient and somewhat dead. Nonetheless, it seems to be delivering a surprising number of good results.

Advertisements

Entry filed under: Agile Project Management.

Slides from “Refactoring Towards Sanity” LiskovCheck: Semi-Automatic Liskov Substitution Principle Adherence Check

4 Comments Add your own

  • 1. Joseph Wilk  |  June 20, 2010 at 4:46 am

    Interesting points.

    Are your QAs happy with the amount of manual testing they have to do? Do they get feed up of the same old things breaking? How much of a gain would there be from releasing more than 2 times a week? In those 2 releases do you measure the number of bugs that get shipped?

    “doing TDD in legacy situations is a tall order” -> Is this more a cultural barrier? I cannot imagine working on a legacy system and not writing tests. Specially tests that are related to areas of the code I have to work on, rather than sprinkling tests everywhere. If anything I need TDD more in legacy systems than I do working on green field projects.

    Michael Feathers book on ‘Working Effectively with Legacy Code’ introduces great ways to test legacy systems and what to test.

    Reply
    • 2. reppel  |  June 20, 2010 at 9:09 am

      I feel that it’s mostly about those “cultural barriers”: TDD does require a way of thinking about writing code which is quite different from the “norm”. The problem I see is that when faced with enhancing or bug-fixing in a legacy situation, you often have to do some refactoring to break dependencies before tests can be written. More than writing the tests themselves, it is this refactoring which is “scary”: It’s seen as taking a lot longer than just doing the fix without tests. In addition, there is probably reluctance to touch working code without dire need, i.e. the refactoring is seen as risky. Based on everybody’s long experience of working the “traditional” way without TDD and refactoring tools and with long release cycles, both concerns are valid. It takes a cultural shift and a lot of learning new things until folks feel comfortable with this. I have had some good experiences with using the wrappering and sprouting approaches described in Michael Feather’s book: We had some situtations where these techniques made for a gentle introduction to TDD because it minimized the need to touch existing code, thereby making both developers and project managers more comfortable with the process. I hope that as familiarity with TDD tools & techniques grows there will be bolder measures.

      QA happiness with the amount of manual testing: That’s directly related to how many QA resources are available. If there were enough testers to get it done, they seemed to be OK with it because the workload is acceptable. This is of course an expensive proposition. This is why I feel that “Zombie Agile” is a kind of transitional step: In the sense that it’s flexible enough to deliver good end user experiences in a fairly predictable manner, it’s ahead of straight waterfall and a whole bagful of hit-and-miss approaches. However, it’s very inefficient.

      I can’t imagine that there is anything to be gained from releasing more than twice a week when manually testing a non-trivial system. There seems to be a threshold beyond which the regression test effort goes ballistic while not necessarily finding all that many new issues. I guess it’s some function of complexity / no. of developers coding / no of QA personnel / frequency of releases.

      Reply
      • 3. Sean  |  August 18, 2011 at 7:11 am

        As the QA manager for one of the teams I can comment on the test side of the equation.

        Were we happy with the amount of manual testing that had to be done? As Robert mentions, it was directly related to the amount of resources available at any given time. When we were short resources because other projects temporarily “stole” them, the manual testing became a burden. When we had a full compliment, the manual testing was completely acceptable.

        There is a cultural component here as well though. The individual testers on the projects were considerably more comfortable with the amount of manual testing than I was. In fact, when we started implementing automation along the way we found that some testers had a lack of trust that it was being done well. Just as the developers and project managers need to grow into a level of comfort with TDD, the testers need to do the same. Small steps along the way (introducing automated integration tests driven by developers as part of their TDD) certainly helps with this transition. Of course, this adds to the expense and inefficiency of doing things and it drives me nuts because I can see the inefficiency.

        Did we get fed up with the same old things breaking? Actually, this was not much of a problem. A skilled development team added to the complexity of the system under test helped to reduce this. The dev team, for the most part, understands the software and the systems fairly well. Because of this the number of new defects introduced and the number of bounce-backs occurring was lower than I have seen elsewhere and in different projects. That’s the good news. The bad news is that due to the complexity of the system, we simply weren’t able to go as deep in areas with every single build. This meant that we were hitting functional areas and digging into fixed areas on a priority basis and may not have seen all defects in an area every cycle. This is a major problem with the test approach required when all you have is manual testing and constrained resources. And yes, that is somewhat concerning.

        How much of a gain would there have been from more regular drops to QA? Unfortunately, because of the reliance on manual testing in an environment were resources were not plentiful at all times, not much of a gain at all. In fact, we had to refuse builds at certain times to allow us to finish testing what we had at a given time. The complexity of the system meant that we were and are, unable to do a complete test pass in a short cycle without a huge number of manual testers. This number of testers was not and will not be forthcoming. We settle with focusing our testing on areas of the code that have been altered (defect correction, new features, feature updates) and then on rotating areas based on risk prioritization. As resources become constrained, the focus is almost entirely on component testing (altered code) which leaves us trying to do a regression cycle near the end a’la waterfall. Deploying hourly or even daily would actually have been more of an impediment to the requirements of the test cycle than they would have been a boon.

        Did we measure the number of bugs shipped in each build? Yes, but as you can likely guess from the test approach outlined, we were not doing this based on a full pass each build and deployment. As Robert mentions we used the information gleaned to make incremental changes to the development process and to focus development and test efforts for the next cycle. Through this analysis we were able to direct the automation that was created which led to an increased level of comfort for the testers and project managers.

        Obviously the approach is not perfect, but, it satisfies the primary business requirement which is to deliver desired functionality with an acceptable level of quality on a regular and understood timeline. I do see this hybrid approach as being a good stepping stool along the way to better things.

  • 4. reppel  |  June 20, 2010 at 1:49 pm

    P.S. Yes, we did measure the number of bugs in each of the releases, with special attention to bounce-backs (claimed fixed, but wasn’t) and, to the extent that we could see what caused them, side effects. I saw both types of bugs as indicators for areas where refactoring and putting them under test might add the most value.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Twitter

   @robertreppel

%d bloggers like this: