Archive

Posts Tagged ‘tfs’

Team Foundation Server Has Great Traceability Until You Need to Report On It

2011/06/19 4 comments

This week I’ve been working on creating our own custom test case coverage and requirement to test case correlation matrix reports based on data in Team Foundation Server (TFS) 2010.

We’re finding TFS reasonably useful at work, particularly with its ALM single-silo approach to data storage and traceability from requirements to releases.  One thing missing out-of-the-box is traceability reporting.  Though TFS does traceability reasonably well it has rather feeble ways of actually reporting it.   The default UI does allow you to trace 1st level links, say requirements to scenarios; it doesn’t provide a way to trace 2nd level or higher links – that is the ability to trace from requirements to scenarios and then on to tasks and test cases in one query.

The other problem is that there are no test case coverage reports or any form of correlation matrix reports.

Never fear because TFS has a rather nice API for pulling data out in your own programs during which you can make your own reports.  We decided to output .CSV files for all the coverage and correlation stuff and .RTF files for the requirements report.  The latter shows a complete list of requirements hi-lighting those that have changed, are new or were deleted (within a certain time interval).

Advertisements
Categories: Development Tags:

Agile is Nice Until You Need to Test Your Software

2011/05/28 3 comments

We recently started to use an agile process model with Team Foundation Server 2010 (TFS).  TFS seems to going rather well but I’m not too sure about the agile model; this is not the fault of TFS of course since agile is independent of TFS and TFS is independent of process models as a whole.  The problem is to do with requirements and testability concerning the system we are developing.  Now,  unlike CMMI, agile does not really have the notion of a pure requirement; instead agile has a user story.  By a pure requirement, I mean that a requirement should independently describe functionality without being tightly coupled with a system action/verb/use case, actor and context.  (That’s why CMMI has use cases).

Unfortunately this is exactly what a user story is – it is the coupling of functionality, actor and context.

Due to this interesting marriage, our quality assurance (QA) teams have confirmed that such a thing is untestable.  For one, user stories are too wordy. Unlike CMMI requirements, agile user stories are made up of not a single line but rather several paragraphs!  Imagine you are only allowed to test a single feature in one paragraph in your home mortgage contract – you can’t, there is just far too much information to be covered by a single test. This is the first arguable short-sight of agile in an dev environment where a reasonable level of QA is being performed.   The same thing would happen in the world of CMMI if the business analyst (BA) were to pass his interview notes in paragraph form straight to developers instead of producing atomic requirements.

Even if the user story were short consisting of a single line similar to CMMI, the structure or grammar of user stories makes it hard to test due to the above-mentioned fusing of functionally, actor and context.

i.e.

As a <type of user> I want <some goal> so that <some reason>”

This is what I refer to as a scenario.  You could have the same thing in CMMI if had a use case UC.01 and two actors User and Admin.  From this example two scenarios could be derived – the scenario where the User invokes UC.01 and the other where Admin invokes UC.01.  Its the same functionality in both scenarios, however the context is different due to the different actors.  Imagine it visually by drawing an ellipse that encloses UC.01 and User and another ellipse around UC.01 and Admin.  In any event, QA would most likely have just a hard a time testing a CMMI system in this fashion if only scenarios and no requirements were provided.

Therefore the same is arguably true for user stories.

Needless to say I’m being a complete TFS geek at the moment! 😉

Categories: Development Tags: , ,

Who Said Team Foundation Server is Only for Windows?!

2011/05/28 Leave a comment

I thought so too until I read this post in the MSDN Social Forums and this relating article about Microsoft Pathways about Team Explorer Everywhere (formerly TeamPrise of SourceGear fame).  With it you gain access to TFS bug tracking, work item tracking, build, reporting and source control from Linux and Mac OS X. It consists of quite a few odds and ends including plug-ins for Eclipse-based IDEs.

When used with the Team Foundation Server Build Extensions Power Tool, one can have a TFS-integrated Java build server using Ant or Maven.

A must read for all cross-platform dev houses!

Categories: Development Tags:

Making a Work Item Field Read-Only After It Has Been Saved in Team Foundation Server 2010

2011/05/09 Leave a comment

There is arguably a need for frequently making many fields in work items read-only after the work item has been created and saved. For example you may want to ensure the description field is read-only to all but project managers; bug priority; requirement title and so on.

After hunting about I came up with the following easy method. Before proceeding, ensure you have downloaded the latest Team Foundation Server Power Tools as it includes the Process Template Editor.

In my example below I am making the Task’s Original Estimate field read-only after saving.

image

image

This basically ensures two things:

  1. That something be entered
  2. That while the user can technically change the value, that unless it matches the original value, that changes to the work item can not be saved

Let’s try it by attempting to edit the pre-saved field:

image

…changing the field (not the warning yellow colour)

image

image

…hmmm ok, let’s type that zero again:

image

Well, what about if we clear the field?

image

image

Yay! Works a treat.

Remember, that TFS generally does not allow you to change the process template of an existing project. However, TFS allows you to export/download the projects template to disk where you may edit it in the Process Editor or the XML editor of your choice and then upload it to TFS. However you will need to recreate your project for it to utilise the new template.

image

Alternatively, the Process Editor can directly edit a work item type in a project on the server. In this mode you can only work on one work item type at a time.  Take great care here!

image

Categories: Development Tags:

Just Because You Can Do a Thing, Does not Mean You Should Do a Thing

2011/04/25 Leave a comment

Microsoft Test Manager 2010 (MTM), the testing and lab management tool extension to Microsoft’s ALM product – Team Foundation Server 2010, is perhaps an example of bad user interface design.  I find the look and feel to be somewhat foreign to user’s familar with Visual Studio but also arguably anything else that Microsoft has made historically.

NewImage

One of my personal pet hates is nested tabbed windows.  I know Mr Cooper (the author of the fine About Face series of books) has covered these types of bad habits before. I find such a GUI style leads to me ultimately getting lost in nested windows.  It’s very difficult to discover or locate functionality.

Microsoft Test Manager excels at not only nested tabbed windows that makes using this tool difficult, but also because it attempts to realise an inductive user interface design (an oxymoron with nested tabs) and fails.

Is that a Title Bar or a Menu?

I was reading an article somewhere on the net when someone mentioned Microsoft Lab Manager – an included tool with Test Manager. Apparently this tool was for running tests on applications particularly those running in virtual machines.  Lab Manager? Where was that? I’d not seen it.  I looked at Test Manager again, here is the upper left of the window:

NewImage

Well nothing conspicuous there, some navigation buttons and a window title…or was it? Lets expand out the view:

NewImage

I’d never noticed that inverted triangle widget between “Testing Center” and “Plan“, clicking on it displays a drop-down for the Lab:

NewImage

Clicking on it makes the application switch to the Lab Center.  Now this brakes several important rules about GUI design.

  1. Menus should never be bang menus – that is they should not act like buttons
  2. All buttons should be captioned with verbs not captioned with the current state.  Many people fall into this trap with single toggle buttons that toggle application state but also toggle the label with “Started” when the application is in a running state; “Stopped” when. This “Testing Center” widget here is in effect a toggle button that is displaying state. Instead it should be a menu that says “Mode” with a good old-fashing ticked “Testing Center” or “Lab Center” as appropriate.  Just like clicking “Start” to shutdown windows, clicking “Testing Center” to switch to “Lab Center” makes no sense
  3. It is not immediately apparent that this is a clickable widget to begin with. Silly people like me mistake it for a window’s title. The inverted triangle should be closer

Lab mode looks like this, I particularly liked how it suddenly seemed empathic towards me by turning green. And no, it wasn’t the result of “envy”.

NewImage

It is perhaps an amateurish piece of software, design-wise at least; I wouldn’t be surprised it was designed by upstart, Windows Presentation Foundation (WPF) developers with little or no design experience.  It’s like:

“…hey! we have this kewl new WPF tech. let’s put it in the testing suite of TFS 2010…”.

This is arguably a formula for a disaster.  Just because you can do a thing does not mean you should.  Take Windows Phone 7. It was done in Silverlight with the XNA team but they forgot some of the most important things in a smartphone such as the ability to copy and paste.  Amateurs.  I pitty the ex PocketPC and Phone Mobile 6 developers at Microsoft.

Another dreadful feature of Test Manager is that of course you can not for the most part look, edit or review items that are defined in Lab mode or vice versa – you must switch modes first.

Oh did I mention there are no popup windows either?  Too bad if you need to look at two things at once.

Amateurish!

 

Categories: Development Tags: , , ,

Continuous Deployment in Team Foundation Server 2010 – It Doesn’t “Just Work”

2011/04/23 Leave a comment

I finally got TFS to deploy my app via a custom TFS build workflow definition to a test environment running in Hyper-V.  It took me about three days to get it going.  I’m yet to know if its all worth it in the end.  Some serious cons I encountered along the way one being that its considerably more difficult to link to TFS “HQ” should your VM not be in the same domain.  Why does MS insist in using NTLM all the flippin time is beyond me.  A test environment will not necessarily have anything of importance we would arguably want secret from the world and locking it up is questionable.

Also placing a VM machine on the domain with machine names generated by Lab Manager with half a GUID in them is sure to annoy the Infrastructure guys sitting near me.  I already had to duck the squash rackets being thrown at me when I mentioned that we need SharePoint 2010 for artefact storage for our ALM projects.  Just kidding guys 😉

The need to deploy Lab, Test and Build agents and all the baggage they come with – just about all the prerequisites for our software!  Just to enable the feature of “‘copying code from the build server to the VM” requires TFS build components on the VM. Ugh. Talk about code bloat.    Well I suppose it saves us having to worry about our product installers having to do it.

A colleague at work once said to me he likes the tools that are “xcopy”-able and don’t install services to minimise the impact on the system.  After looking at TFS CD I can see the wisdom of his statement for TFS is far from it.

Needless to say it required multiple creation and tear downs  of Hyper-V environments. Microsoft System Center Virtual Machine Manager must be the most buggiest piece of excrement to come along since oh Office 2010. VMM can’t even handle me renaming an ISO in the library.  Why does it copy by default anyway. Seems inefficient to me. SCVMM takes ages to realise that I shutdown the VM in Hyper-V Console.  An obvious embarrassment for Microsoft when you consider that such a tool is designed to be used in large scale IT deployments.  And what’s that black bar each time I right-click in the window?  I think it’s a shy context menu who wants to remain hidden.

I don’t think we’ll be leaving VMWare non too soon.

Just like SharePoint 2010, TFS continuous deployment is just as tedious and overly complex to setup.  Reminds me of all the Oracle stories.

This is as far from “it just works” as you can get.