James Newkirk

Agile Conference in Toronto

I will be attending the agile conference in Toronto August 4th – 8th. If you are interested in the details click here. The last information that I received stated that they are expecting almost 1600 people. A far cry from the first XP Universe in 2001. I think we were lucky if we had a 100.

This year I will be doing 2 sessions. Here are the details:

• Rethinking Unit Testing: xUnit.net with Brad Wilson, Tuesday August 5, 2008 at 10:45am – Brad and I wrote xUnit.net and this session will detail the motivations and decisions that went into the design and implementation of the tool.
• TDD Clinic: NUnit, Friday August 8, 2008 at 8:30am – This session is about using NUnit when doing test-first programming.

If your attending and want to chat please contact me. 

Lessons Learned in Programmer Testing

I have just finished the early summer conference circuit. I  spoke at the p&p Summit in Quebec City in early May, TechEd Developer in Orlando, and the Better Software Development Conference in Las Vegas.

The session that I gave at all of the conferences was about lessons that I have learned writing programmer tests over the years. Some of these lessons are now embodied in xUnit.net. I have attached the presentation (2135.7K) to this blog post. You can also download all of the sample code  (214.0K) from the sessions as well. The code is licensed under the Microsoft Public License.

Joe White has a detailed write-up on his blog outlining the TechEd session. He takes me to task about being dogmatic in a couple of places. I will admit to being dogmatic when it comes to test readability and maintenance. The specific lessons “Don’t use Setup and Teardown” and “Don’t use abstract base test classes” are discussed with readability and maintainability as the primary goals. This does potentially increase code duplication but I am willing to do that to improve the communication of the code.

Replacing ExpectedException in NUnit

Many people have heard and read my rants on ExpectedException in the past few months. If you have not heard it  in short I don't believe you should use it. In fact, in xUnit.net there is no such attribute. However, people tell me great; I am not using xUnit.net and I still want to adopt the approach. Well there is good news on a couple of fronts. First, NUnit 2.5 which is currently in alpha has implemented something similar to Assert.Throws. Secondly, if you don't want to wait for NUnit 2.5 or you don't want to upgrade I have provided an implementation of Assert.Throws, Assert.DoesNotThrow, and Record.Exception that you can download from here.

 

One of the biggest flaws of ExpectedException at the method level is that it can often succeed for the wrong reasons. For example, read through the following code and think about whether the test passes or not.

 

[Test]
[ExpectedException(typeof(ArgumentException))]
public void DepositThrowsArgumentExceptionWhenZero()
{
    CheckingAccount account = new CheckingAccount(0.00);

    account.Deposit(0.00);
}

Here is the relevant CheckingAccount code:

public CheckingAccount(double balance)
{
    if (balance == 0) 
       throw new ArgumentException("Initial balance cannot be zero");

    this.balance = balance; 
}

public void Deposit(double amount)
{
    if (amount == 0) 
       throw new ArgumentException("Deposit amount cannot be zero");

    balance += amount;
}

If you were to run this test in NUnit it would succeed. However if you look at the name of the test, DepositThrowsArgumentExceptionWhenZero it succeeds because the creation of the CheckingAccount class with an initial deposit of $0.00 also throws ArgumentException. A better way, in my opinion, uses the XunitAssert.Throws method to isolate the specific piece of code that throws the exception. Here is the same example using XunitAssert.Throws:

[Test]
public void DepositThrowsArgumentExceptionWhenZero()
{
    CheckingAccount account = new CheckingAccount(0.00);

    XunitAssert.Throws<ArgumentException>(
        () => account.Deposit(0.00));
}

If you were to run this test the test would fail on construction of the CheckingAccount object. The only “flaw” in this approach and I mean flaw lightly is that XunitAssert.Throws does violate the 3A pattern. In the downloaded code there is an alternative called Record.Exception which facilitates the writing of the tests using 3A. Here is the same example using Record.Exception.

[Test]
public void DepositThrowsArgumentExceptionWhenZero()
{
    CheckingAccount account = new CheckingAccount(0.00);

    Exception exception = 
        Record.Exception(() => account.Deposit(0.00));

    Assert.IsInstanceOfType(typeof(ArgumentException), exception);
}

I have described the above solutions using the .NET 3.5 Lambda syntax but all of the methods will work with a delegate and there are non-generic implementations as well. The sample code is licensed under the Microsoft Public License.

Download assert.zip (17.2K). I have tested the code with NUnit 2.4.6 as well at the testing tool in Visual Studio 2008 and the sample code includes the tests as well.

xUnit.net 1.0 Released Today

Brad Wilson and I released xUnit.net 1.0 RTM today.

Acknowledgements

Many people have helped us get to our 1.0 release.

First, we want to acknowledge the inventors of the core ideas behind xUnit frameworks, especially Kent Beck and Eric Gamma, without whom we probably wouldn't have anything to talk about today. :)

Second, we want to thank the people who hatched the idea of a new framework with us, including Peter Provost, Brian Button, Scott Densmore, Jonathan Wanagel, Jamie Cansdale, and Drew Miller.

Lastly, our community provided significant feedback to us during the development process by providing feedback on the web site, opening feature requests, reporting bugs, and generally helping guide the design of the framework. We especially want to thank Matt Podwysocki, Ben Hall, Harry Pierson, Scott Hanselman, Kirk Viehland, Daniel Cazzulino, Oren Eini, Jeff Brown, and Charlie Poole.

Looking Back

We release Beta 1 of our framework September 19, 2007, almost exactly 7 months ago. It took a lot longer than we thought it would to get to 1.0, especially given that we had been using the framework extensively internally for a while before that.

Our original release plans called for just a console runner, but our users quickly educated us on the value of variety. We personally are TestDriven.net users, so that became our second runner; our community quickly followed on asking us to support ReSharper and to write a stand-alone GUI runner. Our time spent with the console runner in an automated environment then drove us to write an MSBuild task which provides much better feedback on the testing process.

Extensibility was a very important feature for us. We felt that there was a lot to be desired in the landscape of extensibility for test frameworks. An important early decision we made was to push as much of our "functionality" as we possibly could out into extensions so that we were forced to exercise and improve our extensibility points.

Another unexpected issue was around test runners and their tight coupling to the version of xUnit.net that they were built against. We made a decision to push out the 1.0 release in order to better support the idea of having test runners that were independent from any specific version of xUnit.net. We succeeded for all of our runners, save the ReSharper runner, for reasons we've already documented.

Looking Forward

Our near-term focus is going to be on refining the process of actually using the framework for Test Driven Development, especially in conjunction with Visual Studio. While tools like TestDriven.net provide a lower friction environment than others, we still believe that there is room for improvement in the day to day usage for TDD.

A lot of this work will be centered around the GUI runner. What we released in 1.0 is essentially a bare-bones runner that we labeled experimental. While we would like to know if you have issues with the runner, please be aware that we are intended to make dramatic changes to it to help support our idea of zero-friction TDD.

It is our intention, therefore, to release interim drops of the GUI runner as we make progress, hopefully on a fairly regular schedule.

Additionally, there are several releases on the horizon (especially ReSharper 4.0 and ASP.NET MVC) which will likely also result in new point releases of the framework.

For the longer term (version 2.0), what we really want now is to let the framework get greater adoption so that many more users can help provide feedback on the things they love and the things they don't, the things that just work and the things that could be tweaked. If you download and use 1.0, please visit our forums and tell us about your experiences!

Brad Wilson and I released xUnit.net 1.0 RTM today.

Acknowledgements

Many people have helped us get to our 1.0 release.

First, we want to acknowledge the inventors of the core ideas behind xUnit frameworks, especially Kent Beck and Eric Gamma, without whom we probably wouldn't have anything to talk about today. :)

Second, we want to thank the people who hatched the idea of a new framework with us, including Peter Provost, Brian Button, Scott Densmore, Jonathan Wanagel, Jamie Cansdale, and Drew Miller.

Lastly, our community provided significant feedback to us during the development process by providing feedback on the web site, opening feature requests, reporting bugs, and generally helping guide the design of the framework. We especially want to thank Matt Podwysocki, Ben Hall, Harry Pierson, Scott Hanselman, Kirk Viehland, Daniel Cazzulino, Oren Eini, Jeff Brown, and Charlie Poole.

Looking Back

We release Beta 1 of our framework September 19, 2007, almost exactly 7 months ago. It took a lot longer than we thought it would to get to 1.0, especially given that we had been using the framework extensively internally for a while before that.

Our original release plans called for just a console runner, but our users quickly educated us on the value of variety. We personally are TestDriven.net users, so that became our second runner; our community quickly followed on asking us to support ReSharper and to write a stand-alone GUI runner. Our time spent with the console runner in an automated environment then drove us to write an MSBuild task which provides much better feedback on the testing process.

Extensibility was a very important feature for us. We felt that there was a lot to be desired in the landscape of extensibility for test frameworks. An important early decision we made was to push as much of our "functionality" as we possibly could out into extensions so that we were forced to exercise and improve our extensibility points.

Another unexpected issue was around test runners and their tight coupling to the version of xUnit.net that they were built against. We made a decision to push out the 1.0 release in order to better support the idea of having test runners that were independent from any specific version of xUnit.net. We succeeded for all of our runners, save the ReSharper runner, for reasons we've already documented.

Looking Forward

Our near-term focus is going to be on refining the process of actually using the framework for Test Driven Development, especially in conjunction with Visual Studio. While tools like TestDriven.net provide a lower friction environment than others, we still believe that there is room for improvement in the day to day usage for TDD.

A lot of this work will be centered around the GUI runner. What we released in 1.0 is essentially a bare-bones runner that we labeled experimental. While we would like to know if you have issues with the runner, please be aware that we are intended to make dramatic changes to it to help support our idea of zero-friction TDD.

It is our intention, therefore, to release interim drops of the GUI runner as we make progress, hopefully on a fairly regular schedule.

Additionally, there are several releases on the horizon (especially ReSharper 4.0 and ASP.NET MVC) which will likely also result in new point releases of the framework.

For the longer term (version 2.0), what we really want now is to let the framework get greater adoption so that many more users can help provide feedback on the things they love and the things they don't, the things that just work and the things that could be tweaked. If you download and use 1.0, please visit our forums and tell us about your experiences!

patterns & practices Summit

It time for another patterns & practices Summit. This time the summit will be in Quebec City, Canada, May 6-8th. If you are interetsed in attending click here for details. I will be presenting two sessions.

• Day 3 Keynote - I have some thoughts percolating on what to talk about. The one that keeps coming up the most is around simple design.
• Lessons Learned in Programmer Testing - This session covers some of the key lessons learned using NUnit over the past 5 years.

I will be posting the presentations and any source code here once the event is completed. I hope to see you there.

xUnit.net v1.0(RC2) Released

Brad Wilson and I released xUnit.net V1.0(RC2) today. Brad has a great summary on his blog.

You can download the release from CodePlex. Click here for the release page.

Link Problems

I have corrected the problems with the links to the NUnit Converter and the supplemental content from the Test-Driven Development in Microsoft.Net book.

Click here for the NUnit Converter V1.1.

Click here for the supplemental content from Test-Driven Development in Microsoft.NET.

xUnit.net RC1 Released Today

Brad and I just put the finisihing touches on xUnit.net RC1 today. See Brad's blog post for details.

If all goes well we expect to release v1 sometime in January.

xUnit.net Beta 2 released

On Thursday Brad and I released Beta 2 of xUnit.net. Brad wrote a forum post highlighting the changes that we made going from Beta 1 to Beta2.

In addition we also added support for Resharper in addition to TestDriven.net.

[Test] becomes [Fact]
The rationale for making this change stretches back to earlier this year when I attended the Agile Open Northwest conference. Kent Beck was discussing the paper from David Saff about theories and someone in the audience said that if you were going to call "for-all" tests "Theory" then you should call a single test "Fact". Since then I have had a number of conversations with people and Fact seems to fit.

I would like to thank all the people who have provided us feedback since we launched.

xUnit.net Beta 2 arriving shortly

Brad and I have been busy incorporating the initial feedback from the first beta of xUnit.net. Some of the highlights include support for Resharper 3.0 and a better way to do Fixture SetUp and Teardown. Brad has written up the the details here. We should be releasing the new beta sometime in the next week.