James Newkirk

Link Problems

I have corrected the problems with the links to the NUnit Converter and the supplemental content from the Test-Driven Development in Microsoft.Net book.

Click here for the NUnit Converter V1.1.

Click here for the supplemental content from Test-Driven Development in Microsoft.NET.

xUnit.net RC1 Released Today

Brad and I just put the finisihing touches on xUnit.net RC1 today. See Brad's blog post for details.

If all goes well we expect to release v1 sometime in January.

xUnit.net Beta 2 released

On Thursday Brad and I released Beta 2 of xUnit.net. Brad wrote a forum post highlighting the changes that we made going from Beta 1 to Beta2.

In addition we also added support for Resharper in addition to TestDriven.net.

[Test] becomes [Fact]
The rationale for making this change stretches back to earlier this year when I attended the Agile Open Northwest conference. Kent Beck was discussing the paper from David Saff about theories and someone in the audience said that if you were going to call "for-all" tests "Theory" then you should call a single test "Fact". Since then I have had a number of conversations with people and Fact seems to fit.

I would like to thank all the people who have provided us feedback since we launched.

xUnit.net Beta 2 arriving shortly

Brad and I have been busy incorporating the initial feedback from the first beta of xUnit.net. Some of the highlights include support for Resharper 3.0 and a better way to do Fixture SetUp and Teardown. Brad has written up the the details here. We should be releasing the new beta sometime in the next week.

The CodePlex Team and the patterns & practices Summit

The CodePlex team will be well represented again at the patterns & practices Summit in Redmond, WA - November 5-9, 2007 . We will be presenting or co-presenting the following sessions:

• "Yet Another Talk on Agility" - Brad Wilson and Peter Provost
• Dependency Injection Frameworks - Brad Wilson and Scott Densmore
• EntLib Devolved - Scott Densmore
• Make It Your Own - Scott Densmore
• Lessons Learned in Programmer Testing - Jim Newkirk
• CodePlex is sponsoring an evening event on Tuesday titled "Open Source in the Enterprise".

Peter's not on the CodePlex team but we like him just the same. For additonal information on the summit and all of the sessions please click here.

Announcing xUnit.net

In the 5 years since the release of NUnit 2.0, there have been millions of lines of code written using the various unit testing frameworks for .NET. About a year ago it became clear to myself and Brad Wilson that there were some very clear patterns of success (and failure) with the tools we were using for writing tests. Rather than repeating guidance about "do X" or "don't do Y", it seemed like it was the right time to reconsider the framework itself and see if we could codify some of those rules.

 

Additionally, the .NET framework itself has evolved a lot since its v1 release in early 2002. Being able to leverage some of the new framework features can help us write clearer tests.

 

Another aspect of change that we wanted to affect was bringing the testing framework more closely in line with the .NET platform. Many of the decisions we made, which we enumerate below, were driven by this desire. We wanted an architecture which is built specifically for programmer testing (specifically Test-Driven Development), which can also be very easily extended to support other kinds of testing (like automated acceptance tests).

 

Finally, there have been advances in other unit test library implementations that have not really surfaced in the .NET community.

 

While any one of these reasons would not necessarily have been sufficient to create a new testing framework, the combination of them all made us want to undertake a new project: xUnit.net.

 

Lessons Learned

 

   
• Single Object Instance per Test Method. Much has been written about why this improves test isolation. In xUnit.net we create a new instance per test. For more information, see http://blogs.msdn.com/jamesnewkirk/archive/2004/12/04/275172.aspx and http://martinfowler.com/bliki/JunitNewInstance.html.
   
• No [SetUp] or [TearDown]. I blogged recently about some of the problems related to SetUp/TearDown. xUnit.net does not have any built-in support for this capability. For more information, see http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html.
   
• No [ExpectedException]. Rather that decorating a method with an attribute, we have returned to the old JUnit style of Assert.Throws for expected exceptions. This helps two major issues: 1. With [ExpectedException] it's possible to hide real errors when the wrong method call throws an exception, and 2. Allows your tests to continue to obey the Arrange-Act-Assert pattern (as coined "3A" by William Wake).
   
• Aspect-Like Functionality. End users extended NUnit and MbUnit with cross-cutting concerns that could be attached to test methods (an example is automatically rolling back changes made to a database during the test). This made the tests simpler to write and allowed more consistent usage of the cross-cutting operations. xUnit.net makes it very simple to create such operations and attach them to test methods.
   
• Reducing the Number of Custom Attributes. Sometimes, the excessive use of attributes can make you feel like you've diverged far from the underlying language. xUnit.net removed some attributes from the framework, instead relying on language features to provide similar functionality:      
        
  • [TestFixture] was removed entirely, so tests can be anywhere.
         
  • [Ignore] is expressed using the Skip= parameter on [Test].
         
  • [SetUp] and [TearDown] are removed.
         
  • [ExpectedException] was replaced with Assert.Throws.
         
  • [TestFixtureSetup] and [TestFixtureTearDown] are instead expressed as implementations of an interface (ITestFixture).
         
  • Support for IDisposable was added for tests.
      
     

 

Language Features

 

   
• Use of Generics. The addition to generics to .NET 2.0 allowed much more concise assertions, even allowing some compile-time errors (mismatched types) to be caught. It also allowed us to add type-specific comparer support for the more common asserts (like Equal and NotEqual).
   
• Anonymous Delegates. Support for anonymous delegates in .NET 2.0 made the syntax for Assert.Throws much more compact and readable. Here are two examples of Assert.Throws:      
        
  Assert.Throws<InvalidOperationException>(delegate { operation(); });  // .NET 2.0      
        
  Assert.Throws<InvalidOperationException>(() => operation());  // .NET 3.5

 

Test Runner

 

   
• For v1, we are shipping a console-based test runner, but not a GUI-based test runner. Based on user feedback, we will determine which other runners xUnit.net should support in future releases.

 

Extensibility

 

   
• Assert extensibility. Through the use of custom comparers (that implement IComparer<T>), you can extend the concepts of Equal, NotEqual, InRange, and NotInRange for your tests. There are two examples of this in the Samples project (one which does case-insensitive comparisons and one which does date-only DateTime comparisons).
   
• Test method extensibility. The definition of how to run a test method can be extended. There are two example of this: the first, in the extensions DLL, is the [Theory] attribute which allows data-driven tests; the second, in the samples, is the [RepeatTest] attribute which runs a test method multiple times in a row. For more information on data theories, see http://shareandenjoy.saff.net/2006/12/new-paper-practice-of-theories.html.
   
• Test class extensibility. The definition of run to run a test class can be extended. There is an example of this in the extensions DLL, the [RunWithNUnit] attribute which allows you to have mixed xUnit.net and NUnit tests in the same assembly, all executable by any xUnit.net runner.

Ridiculous Legalese

I don't know what it is but every time I have my car in for service I get the urge to blog something. This time something was fixed on my car and the person at the cashier wanted me to sign something inidcating that I understood what the Limited Warranty was for the part was fixed. An excerpt from the Limited Warranty is as follows:

Dealer warrants all labor with respect to the repairs performed under the original repair order against defects in workmanship under normal use for a period of 24 months or unlimited miles, whichever comes first. ...

It seems to me that it just could have said 24 months because I believe 24 months will be here before unlimited miles.

Why you should not use SetUp and TearDown in NUnit

When I lead the team that built NUnit we implemented SetUp and TearDown similar to the way they were implemented in JUnit. For those who do not know SetUp and TearDown are attributes on methods in a TestFixture that perform common initialization and destruction.  Here is an example that demonstrates how and when SetUp and TearDown are called,

[TestFixture]
public class TestFixtureLifetime
{
    [SetUp]
    public void BeforeTest()
    { Console.WriteLine("BeforeTest"); }

    [TearDown]
    public void AfterTest()
    { Console.WriteLine("AfterTest"); }

    [Test]
    public void Test1()
    { Console.WriteLine("Test1"); }

    [Test]
    public void Test2()
    { Console.WriteLine("Test2"); }
}

When I run this test  I get the following output: .

BeforeTest
Test1
AfterTest
BeforeTest
Test2
AfterTest

I used to think that factoring out duplicated code into a common SetUp and TearDown method was good idea. My reasoning was based on one of the tenants of Simple Design; Remove Code Duplication, even from your test code. In order to better demonstrate why my thoughts have changed let's look at a slightly modified MoneyTest from the NUnit samples. MoneyTest has 3 Test methods (IsZero, BagAdd, and BagSubtractIsZero) and a Setup method, (BeforeTest). 

[TestFixture]
public class MoneyTest
{
    private Money f12CHF;
    private Money f14CHF;
    private Money f7USD;

    private MoneyBag moneyBag;

    [SetUp]
    public void BeforeTest()
    {
        f12CHF = new Money(12, "CHF");
        f14CHF = new Money(14, "CHF");
        f7USD = new Money(7, "USD");

        moneyBag = new MoneyBag(f12CHF, f7USD);
    }

    [Test]
    public void IsZero()
    {
        Money[] bag = { new Money(0, "CHF"), new Money(0, "USD") };
        Assert.IsTrue(new MoneyBag(bag).IsZero);
    }

    [Test]
    public void BagAdd()
    {
        Money[] bag = { new Money(26, "CHF"), new Money(7, "USD") };
        MoneyBag expected = new MoneyBag(bag);
        Assert.AreEqual(expected, f14CHF.Add(moneyBag));
    }

    [Test]
    public void BagSubtractIsZero()
    {
        Assert.IsTrue(moneyBag.Subtract(moneyBag).IsZero);       
    }
}

The problem that I have with SetUp in this case is twofold. The first and primary complaint is that when I am reading each test I have to glance up to BeforeTest() to see the values that are being used in the test. Worse yet if there was a TearDown method I would need to look in 3 methods. The second issue is that BeforeTest() initializes member variables for all 3 tests which complicates BeforeTest() and makes it violate the single responsibility pattern.  In order to fix this I would remove the BeforeTest() method and place the specific variables in each test - here is the refactored code:

[TestFixture]
public class NoSetUpMoneyTest
{
    [Test]
    public void IsZero()
    {
        Money[] bag = { new Money(0, "CHF"),
                        new Money(0, "USD") };
        Assert.IsTrue(new MoneyBag(bag).IsZero);
    }

    [Test]
    public void BagAdd()
    {
        Money f7USD = new Money(7, "USD");
        Money f12CHF = new Money(12, "CHF");
        Money f14CHF = new Money(14, "CHF");
        Money f26CHF = new Money(26, "CHF");

        Money[] bag = { f26CHF, f7USD };
        MoneyBag expected = new MoneyBag(bag);

        MoneyBag moneyBag = new MoneyBag(f12CHF, f7USD);

        Assert.AreEqual(expected, f14CHF.Add(moneyBag));
    }

    [Test]
    public void BagSubtractIsZero()
    {
        MoneyBag moneyBag = new MoneyBag(new Money(7, "USD"),
                                         new Money(12, "CHF"));
        Assert.IsTrue(moneyBag.Subtract(moneyBag).IsZero);
    }
}

In the refactored code the BeforeTest() is removed and each test is forced to do the initialization for what it needs to run. This makes it very easy to read each test as a single entity. Also, it has the side benefit of making it very clear which tests are more complicated than others. Compare the initialization code needed for BagAdd compared to IsZero. In addition the new class has no member variables. This is critical for test isolation. Without member variables there is no mechanism contained within the class for one test to mess with another.

These reasons lead me to never want to use common setup and tear down methods. In fact, I believe that this feature should be removed from the tool. Some people will argue that you cannot build a tool that stops people from writing poor code. However, you can write a tool that prevents people from shooting themselves in the foot.

NUnit Converter V1.1 - Moved from GotDotNet

Due to the shutdown of GotDotNet I have moved the NUnit Converter V1.1 to here. Note: This is not a new version. If you have comments or questions please let me know.

It's not just about the tool...!

In the past I have used the quote by James Carville "The economy, stupid" who was a strategist on Bill Clinton's presidential campaign in 1992. This quote helped the team focus on one of the core issues in the campaign. I bring this up again because it relates to one of the tenants of the Agile Manifesto. Individuals and interactions over processes and tools.  The focus of this statement was to make it clear that you should prefer individuals and how they interact instead of relying on any specific process or tool. It also says to me that a tool should play a supporting role on a project, not the lead.    With that said, I wanted to introduce a book that I co-wrote with Will Stott. The book is titled, "Visual Studio Team System: Better Software Development for Agile Teams". On the surface you might be saying isn't the title of this post "It's not just about the tool..."? The book does happen to use Visual Studio Team System (VSTS) but the focus of the book is how you do Agile Development on a project from start to finish. In fact, one of the quotes we received from Scott W. Ambler (one of the reviewers of the book) is as follows: "This book provides practical advice for real-world Agile software development. It goes beyond programming to address modeling, deployment, database, and management issues (to name a few) that most Agile development books fail to address. I wish I'd written it." The book contains a large number of examples and hands-on exercises. For more details please see the companion web-sites: http://www.bettersoftwaredevelopment.org/ http://www.codeplex.com/VSTSCode If you have any questions please let me know. Will and I look forward to your feedback.