Home > Development > Agile is Nice Until You Need to Test Your Software

Agile is Nice Until You Need to Test Your Software

We recently started to use an agile process model with Team Foundation Server 2010 (TFS).  TFS seems to going rather well but I’m not too sure about the agile model; this is not the fault of TFS of course since agile is independent of TFS and TFS is independent of process models as a whole.  The problem is to do with requirements and testability concerning the system we are developing.  Now,  unlike CMMI, agile does not really have the notion of a pure requirement; instead agile has a user story.  By a pure requirement, I mean that a requirement should independently describe functionality without being tightly coupled with a system action/verb/use case, actor and context.  (That’s why CMMI has use cases).

Unfortunately this is exactly what a user story is – it is the coupling of functionality, actor and context.

Due to this interesting marriage, our quality assurance (QA) teams have confirmed that such a thing is untestable.  For one, user stories are too wordy. Unlike CMMI requirements, agile user stories are made up of not a single line but rather several paragraphs!  Imagine you are only allowed to test a single feature in one paragraph in your home mortgage contract – you can’t, there is just far too much information to be covered by a single test. This is the first arguable short-sight of agile in an dev environment where a reasonable level of QA is being performed.   The same thing would happen in the world of CMMI if the business analyst (BA) were to pass his interview notes in paragraph form straight to developers instead of producing atomic requirements.

Even if the user story were short consisting of a single line similar to CMMI, the structure or grammar of user stories makes it hard to test due to the above-mentioned fusing of functionally, actor and context.


As a <type of user> I want <some goal> so that <some reason>”

This is what I refer to as a scenario.  You could have the same thing in CMMI if had a use case UC.01 and two actors User and Admin.  From this example two scenarios could be derived – the scenario where the User invokes UC.01 and the other where Admin invokes UC.01.  Its the same functionality in both scenarios, however the context is different due to the different actors.  Imagine it visually by drawing an ellipse that encloses UC.01 and User and another ellipse around UC.01 and Admin.  In any event, QA would most likely have just a hard a time testing a CMMI system in this fashion if only scenarios and no requirements were provided.

Therefore the same is arguably true for user stories.

Needless to say I’m being a complete TFS geek at the moment! 😉

Categories: Development Tags: , ,
  1. martin
    2011/05/28 at 10:38 pm

    Hey Mickey,
    Glad to see a semi agile approach is at least being attempted finally. However it sounds like you are missing some key elements. Most notably acceptance criteria. This is what the developers should be developing against and what testers should be testing against. A user story by itself is just a placeholder (in the case of XP it was said to be a placeholder for a conversation with the user/product owner).
    The criteria can sometimes take the form of:
    “Given , when , then “.
    If this gets to complicated/wordy then look at breaking up the story into smaller pieces.

    Personally, after all of three weeks actual experience with Scrum and TFS, I can say that my opinion is the opposite of yours – scrum is going quite well, but I’m not too sure about TFS 🙂


    • martin
      2011/05/28 at 10:42 pm

      opps the criteria format should read:
      “Given [initial state], when [action taken], then [expected result]”


    • MickyD
      2011/05/28 at 11:47 pm

      Hi Martin,
      Indeed, anything to get away from waterfall. Acceptance criteria was brought up recently actually where we reasoned that AC are in many ways, much like requirements anyway. Some say to put AC on the back of a user story card which I thought was an odd thing to do from the perspective of doing requirements. Oh darn I forgot to turn the card over 😉

      So now we are planning to modify the MS Agile process template in TFS in order to introduce a “requirement” workitem and link it to user stories. We think it might work as it takes the nice ready-friendly user story (actor and context) and fuse it with one or more atomic but testable requirements (functionality). Which pleases model-based testing to no end.

      More than likely there are other ways to do it, time will tell if we can improve on it.

      TFS hey? yeah we don’t have unanimous support either. Just so long as its not Applix 🙂


  1. No trackbacks yet.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: