Creating a NUnity activity for TFS 2010 Build

by Administrator 30. Dezember 2011 04:42

TFS build is really nice - I used to work with CruiseControl.NET / NAnt before and having not to work with angle brackets and debug logs all the time makes your job so much easier. If you stick to Microsoft tools, you'll find that lots of activities for your build are already there, if you want something else, it's really easy to implement your own. In our project we decided to go with NUnit instead of MSTest for several reasons. There is no standard activity to run NUnit tests inside a build process, however the Community TFS Build Extensions project offers such a thing.

I found that there are some things I would like to have different when running Unit tests an decided to roll my own activity for that matter. You'll find the source attached to this post. I do not intend to maintain a project here or contribute to the community project, however I hope you'll find some interesting insights on how to create your own activities.

The first thing that I wanted differently was the ability to run multiple unit tests at once. Since unit tests are independent of each other by definition and most build servers will have multiple cores one should see a great performance boost when doing that. TFS Build is built on top of Windows Workflow Foundation. There is a container that you can drop activities into that should run in parallel, however when placing several NUnit activities in it you'll see that they run sequentially nevertheless. The reason for that is, that workflow foundation has a concept of an idle state in activities. When an activity reaches a point in it's execution where it cannot do anything until something happened (e.g. a download finished, an executable completed...), it signals it's idle state to the workflow runtime and other activities in a container marked for parallel execution may get a time slice.

Activities that support idleness inherit from another base class and use the Begin/End idiom used in .NET. Here is an extract of the entry method of the activity:

protected override IAsyncResult BeginExecute( AsyncCodeActivityContext context, AsyncCallback callback, object state )
{
  // ...Initialization code
  Func<ExecutionContext> execution =
    () =>
    {
      ... here goes the code that takes some time...
    };
   
  context.UserState = execution;
  return execution.BeginInvoke( callback, state );
}

The actual thing the we consider being the idle state is when NUnit runs and we wait for it's return. You will see that the code inside the execution function with the NUnit process being launched:

process.Start();
process.BeginOutputReadLine();
process.BeginErrorReadLine();

The Start() method does not wait for the process to return. To wait, you use the WaitForExit() method. Now let's look at the method that ends the asynchronous call:

protected override bool EndExecute( AsyncCodeActivityContext context, IAsyncResult result )
{
   Func<ExecutionContext> execution = ( Func<ExecutionContext> )context.UserState;
   ExecutionContext executionContext = execution.EndInvoke( result );
   executionContext.Process.WaitForExit();
}

As you see the UserState is used to marshal state in between the two methods. The ExecutionContext is a class by me that holds information about the process being launched.

With this construct in place you will be able to run several NUnit activities in a parallel block. The next thing that somehow bogged me was, that when publishing tests results to the build, you will not see to which project tests belong:

Neither the link to get to the test results, nor the results themselves indicated the test solution. This was a little annoying to me. I started off with the XSLT document from the community project (included with the ZIP) to transform the nunit xml output to the mstest xml format. There is a name attribute on the root of the mstest that contains the name. I injected that via XLinq (I'm not really into XSLT and want it to stay that way :P).

XDocument document = XDocument.Load( memoryStream );
document.Root.Attribute( "name" ).Value = Path.GetFileNameWithoutExtension( Project.Get( context ) );

The above code takes the XSL transformed document and injects the value of the node.

At this point I had everything I needed in place, however I found out that sometimes publishing the results of the tests failed. I did not notice the problem with the community activiy, however I came to the conclusion that the same might happen here - though more seldomly. This article pointed me to the right direction. An MSTest file contains a GUID that needs to be unqiue among tests in a build. If it's not and you publish another test with the same GUID, you will get an error that the results have already been published. Peeking into the XSLT i found out that the GUID is generated from a fixed part and a part that depends on the current time (down to the second). This causes problem when you execute multiple tests runs in the same second - highly unlikely when synchronous, highly likely when asynchronous. I injected a new GUID to prevent clashes.

XDocument document = XDocument.Load( memoryStream );
document.Root.Attribute( "id" ).Value = Guid.NewGuid().ToString();

That's all I wanted to show. Thanks again to the guys that did the community project - I would not have done this without them.

NUnitActivity.zip (7,10 kb)

Tags: , , ,

My Stuff | Projects | TFS

NUnit wih Find Dialog

by Administrator 12. Juli 2009 04:03
I added this little mod to the pages. Enjoy.

Tags: ,

General | My Stuff

About the author

for comments and suggestions contact:

 

Month List