James Newkirk

xUnit.net 1.0 Released Today

Brad Wilson and I released xUnit.net 1.0 RTM today.

Acknowledgements

Many people have helped us get to our 1.0 release.

First, we want to acknowledge the inventors of the core ideas behind xUnit frameworks, especially Kent Beck and Eric Gamma, without whom we probably wouldn't have anything to talk about today. :)

Second, we want to thank the people who hatched the idea of a new framework with us, including Peter Provost, Brian Button, Scott Densmore, Jonathan Wanagel, Jamie Cansdale, and Drew Miller.

Lastly, our community provided significant feedback to us during the development process by providing feedback on the web site, opening feature requests, reporting bugs, and generally helping guide the design of the framework. We especially want to thank Matt Podwysocki, Ben Hall, Harry Pierson, Scott Hanselman, Kirk Viehland, Daniel Cazzulino, Oren Eini, Jeff Brown, and Charlie Poole.

Looking Back

We release Beta 1 of our framework September 19, 2007, almost exactly 7 months ago. It took a lot longer than we thought it would to get to 1.0, especially given that we had been using the framework extensively internally for a while before that.

Our original release plans called for just a console runner, but our users quickly educated us on the value of variety. We personally are TestDriven.net users, so that became our second runner; our community quickly followed on asking us to support ReSharper and to write a stand-alone GUI runner. Our time spent with the console runner in an automated environment then drove us to write an MSBuild task which provides much better feedback on the testing process.

Extensibility was a very important feature for us. We felt that there was a lot to be desired in the landscape of extensibility for test frameworks. An important early decision we made was to push as much of our "functionality" as we possibly could out into extensions so that we were forced to exercise and improve our extensibility points.

Another unexpected issue was around test runners and their tight coupling to the version of xUnit.net that they were built against. We made a decision to push out the 1.0 release in order to better support the idea of having test runners that were independent from any specific version of xUnit.net. We succeeded for all of our runners, save the ReSharper runner, for reasons we've already documented.

Looking Forward

Our near-term focus is going to be on refining the process of actually using the framework for Test Driven Development, especially in conjunction with Visual Studio. While tools like TestDriven.net provide a lower friction environment than others, we still believe that there is room for improvement in the day to day usage for TDD.

A lot of this work will be centered around the GUI runner. What we released in 1.0 is essentially a bare-bones runner that we labeled experimental. While we would like to know if you have issues with the runner, please be aware that we are intended to make dramatic changes to it to help support our idea of zero-friction TDD.

It is our intention, therefore, to release interim drops of the GUI runner as we make progress, hopefully on a fairly regular schedule.

Additionally, there are several releases on the horizon (especially ReSharper 4.0 and ASP.NET MVC) which will likely also result in new point releases of the framework.

For the longer term (version 2.0), what we really want now is to let the framework get greater adoption so that many more users can help provide feedback on the things they love and the things they don't, the things that just work and the things that could be tweaked. If you download and use 1.0, please visit our forums and tell us about your experiences!

Brad Wilson and I released xUnit.net 1.0 RTM today.

Acknowledgements

Many people have helped us get to our 1.0 release.

First, we want to acknowledge the inventors of the core ideas behind xUnit frameworks, especially Kent Beck and Eric Gamma, without whom we probably wouldn't have anything to talk about today. :)

Second, we want to thank the people who hatched the idea of a new framework with us, including Peter Provost, Brian Button, Scott Densmore, Jonathan Wanagel, Jamie Cansdale, and Drew Miller.

Lastly, our community provided significant feedback to us during the development process by providing feedback on the web site, opening feature requests, reporting bugs, and generally helping guide the design of the framework. We especially want to thank Matt Podwysocki, Ben Hall, Harry Pierson, Scott Hanselman, Kirk Viehland, Daniel Cazzulino, Oren Eini, Jeff Brown, and Charlie Poole.

Looking Back

We release Beta 1 of our framework September 19, 2007, almost exactly 7 months ago. It took a lot longer than we thought it would to get to 1.0, especially given that we had been using the framework extensively internally for a while before that.

Our original release plans called for just a console runner, but our users quickly educated us on the value of variety. We personally are TestDriven.net users, so that became our second runner; our community quickly followed on asking us to support ReSharper and to write a stand-alone GUI runner. Our time spent with the console runner in an automated environment then drove us to write an MSBuild task which provides much better feedback on the testing process.

Extensibility was a very important feature for us. We felt that there was a lot to be desired in the landscape of extensibility for test frameworks. An important early decision we made was to push as much of our "functionality" as we possibly could out into extensions so that we were forced to exercise and improve our extensibility points.

Another unexpected issue was around test runners and their tight coupling to the version of xUnit.net that they were built against. We made a decision to push out the 1.0 release in order to better support the idea of having test runners that were independent from any specific version of xUnit.net. We succeeded for all of our runners, save the ReSharper runner, for reasons we've already documented.

Looking Forward

Our near-term focus is going to be on refining the process of actually using the framework for Test Driven Development, especially in conjunction with Visual Studio. While tools like TestDriven.net provide a lower friction environment than others, we still believe that there is room for improvement in the day to day usage for TDD.

A lot of this work will be centered around the GUI runner. What we released in 1.0 is essentially a bare-bones runner that we labeled experimental. While we would like to know if you have issues with the runner, please be aware that we are intended to make dramatic changes to it to help support our idea of zero-friction TDD.

It is our intention, therefore, to release interim drops of the GUI runner as we make progress, hopefully on a fairly regular schedule.

Additionally, there are several releases on the horizon (especially ReSharper 4.0 and ASP.NET MVC) which will likely also result in new point releases of the framework.

For the longer term (version 2.0), what we really want now is to let the framework get greater adoption so that many more users can help provide feedback on the things they love and the things they don't, the things that just work and the things that could be tweaked. If you download and use 1.0, please visit our forums and tell us about your experiences!

patterns & practices Summit

It time for another patterns & practices Summit. This time the summit will be in Quebec City, Canada, May 6-8th. If you are interetsed in attending click here for details. I will be presenting two sessions.

• Day 3 Keynote - I have some thoughts percolating on what to talk about. The one that keeps coming up the most is around simple design.
• Lessons Learned in Programmer Testing - This session covers some of the key lessons learned using NUnit over the past 5 years.

I will be posting the presentations and any source code here once the event is completed. I hope to see you there.

xUnit.net v1.0(RC2) Released

Brad Wilson and I released xUnit.net V1.0(RC2) today. Brad has a great summary on his blog.

You can download the release from CodePlex. Click here for the release page.

Link Problems

I have corrected the problems with the links to the NUnit Converter and the supplemental content from the Test-Driven Development in Microsoft.Net book.

Click here for the NUnit Converter V1.1.

Click here for the supplemental content from Test-Driven Development in Microsoft.NET.

xUnit.net RC1 Released Today

Brad and I just put the finisihing touches on xUnit.net RC1 today. See Brad's blog post for details.

If all goes well we expect to release v1 sometime in January.

xUnit.net Beta 2 released

On Thursday Brad and I released Beta 2 of xUnit.net. Brad wrote a forum post highlighting the changes that we made going from Beta 1 to Beta2.

In addition we also added support for Resharper in addition to TestDriven.net.

[Test] becomes [Fact]
The rationale for making this change stretches back to earlier this year when I attended the Agile Open Northwest conference. Kent Beck was discussing the paper from David Saff about theories and someone in the audience said that if you were going to call "for-all" tests "Theory" then you should call a single test "Fact". Since then I have had a number of conversations with people and Fact seems to fit.

I would like to thank all the people who have provided us feedback since we launched.

xUnit.net Beta 2 arriving shortly

Brad and I have been busy incorporating the initial feedback from the first beta of xUnit.net. Some of the highlights include support for Resharper 3.0 and a better way to do Fixture SetUp and Teardown. Brad has written up the the details here. We should be releasing the new beta sometime in the next week.

The CodePlex Team and the patterns & practices Summit

The CodePlex team will be well represented again at the patterns & practices Summit in Redmond, WA - November 5-9, 2007 . We will be presenting or co-presenting the following sessions:

• "Yet Another Talk on Agility" - Brad Wilson and Peter Provost
• Dependency Injection Frameworks - Brad Wilson and Scott Densmore
• EntLib Devolved - Scott Densmore
• Make It Your Own - Scott Densmore
• Lessons Learned in Programmer Testing - Jim Newkirk
• CodePlex is sponsoring an evening event on Tuesday titled "Open Source in the Enterprise".

Peter's not on the CodePlex team but we like him just the same. For additonal information on the summit and all of the sessions please click here.

Announcing xUnit.net

In the 5 years since the release of NUnit 2.0, there have been millions of lines of code written using the various unit testing frameworks for .NET. About a year ago it became clear to myself and Brad Wilson that there were some very clear patterns of success (and failure) with the tools we were using for writing tests. Rather than repeating guidance about "do X" or "don't do Y", it seemed like it was the right time to reconsider the framework itself and see if we could codify some of those rules.

 

Additionally, the .NET framework itself has evolved a lot since its v1 release in early 2002. Being able to leverage some of the new framework features can help us write clearer tests.

 

Another aspect of change that we wanted to affect was bringing the testing framework more closely in line with the .NET platform. Many of the decisions we made, which we enumerate below, were driven by this desire. We wanted an architecture which is built specifically for programmer testing (specifically Test-Driven Development), which can also be very easily extended to support other kinds of testing (like automated acceptance tests).

 

Finally, there have been advances in other unit test library implementations that have not really surfaced in the .NET community.

 

While any one of these reasons would not necessarily have been sufficient to create a new testing framework, the combination of them all made us want to undertake a new project: xUnit.net.

 

Lessons Learned

 

   
• Single Object Instance per Test Method. Much has been written about why this improves test isolation. In xUnit.net we create a new instance per test. For more information, see http://blogs.msdn.com/jamesnewkirk/archive/2004/12/04/275172.aspx and http://martinfowler.com/bliki/JunitNewInstance.html.
   
• No [SetUp] or [TearDown]. I blogged recently about some of the problems related to SetUp/TearDown. xUnit.net does not have any built-in support for this capability. For more information, see http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html.
   
• No [ExpectedException]. Rather that decorating a method with an attribute, we have returned to the old JUnit style of Assert.Throws for expected exceptions. This helps two major issues: 1. With [ExpectedException] it's possible to hide real errors when the wrong method call throws an exception, and 2. Allows your tests to continue to obey the Arrange-Act-Assert pattern (as coined "3A" by William Wake).
   
• Aspect-Like Functionality. End users extended NUnit and MbUnit with cross-cutting concerns that could be attached to test methods (an example is automatically rolling back changes made to a database during the test). This made the tests simpler to write and allowed more consistent usage of the cross-cutting operations. xUnit.net makes it very simple to create such operations and attach them to test methods.
   
• Reducing the Number of Custom Attributes. Sometimes, the excessive use of attributes can make you feel like you've diverged far from the underlying language. xUnit.net removed some attributes from the framework, instead relying on language features to provide similar functionality:      
        
  • [TestFixture] was removed entirely, so tests can be anywhere.
         
  • [Ignore] is expressed using the Skip= parameter on [Test].
         
  • [SetUp] and [TearDown] are removed.
         
  • [ExpectedException] was replaced with Assert.Throws.
         
  • [TestFixtureSetup] and [TestFixtureTearDown] are instead expressed as implementations of an interface (ITestFixture).
         
  • Support for IDisposable was added for tests.
      
     

 

Language Features

 

   
• Use of Generics. The addition to generics to .NET 2.0 allowed much more concise assertions, even allowing some compile-time errors (mismatched types) to be caught. It also allowed us to add type-specific comparer support for the more common asserts (like Equal and NotEqual).
   
• Anonymous Delegates. Support for anonymous delegates in .NET 2.0 made the syntax for Assert.Throws much more compact and readable. Here are two examples of Assert.Throws:      
        
  Assert.Throws<InvalidOperationException>(delegate { operation(); });  // .NET 2.0      
        
  Assert.Throws<InvalidOperationException>(() => operation());  // .NET 3.5

 

Test Runner

 

   
• For v1, we are shipping a console-based test runner, but not a GUI-based test runner. Based on user feedback, we will determine which other runners xUnit.net should support in future releases.

 

Extensibility

 

   
• Assert extensibility. Through the use of custom comparers (that implement IComparer<T>), you can extend the concepts of Equal, NotEqual, InRange, and NotInRange for your tests. There are two examples of this in the Samples project (one which does case-insensitive comparisons and one which does date-only DateTime comparisons).
   
• Test method extensibility. The definition of how to run a test method can be extended. There are two example of this: the first, in the extensions DLL, is the [Theory] attribute which allows data-driven tests; the second, in the samples, is the [RepeatTest] attribute which runs a test method multiple times in a row. For more information on data theories, see http://shareandenjoy.saff.net/2006/12/new-paper-practice-of-theories.html.
   
• Test class extensibility. The definition of run to run a test class can be extended. There is an example of this in the extensions DLL, the [RunWithNUnit] attribute which allows you to have mixed xUnit.net and NUnit tests in the same assembly, all executable by any xUnit.net runner.

Ridiculous Legalese

I don't know what it is but every time I have my car in for service I get the urge to blog something. This time something was fixed on my car and the person at the cashier wanted me to sign something inidcating that I understood what the Limited Warranty was for the part was fixed. An excerpt from the Limited Warranty is as follows:

Dealer warrants all labor with respect to the repairs performed under the original repair order against defects in workmanship under normal use for a period of 24 months or unlimited miles, whichever comes first. ...

It seems to me that it just could have said 24 months because I believe 24 months will be here before unlimited miles.