Test Driven Development

Test Driven Development promises to improve software quality by pushing developers to think hard about requirements, interfaces and corner cases, create tests early in the development cycle, and provide a test framework that enables and encourages re-factoring.

I decided to give it a try myself while working on one of the labs from my son's Computer Science class at RPI. It was a very simple assignment - write a function that returns the maximum of three integers and test the function with some sample data.  Test Driven Development recommends the following:

1. Write a test.
2. Run the test and verify it fails.
3. Write some code.
4. Run the tests and fix the code until the tests pass.
5. Re-factor the code.
6. Repeat.

For some reason I can't yet explain, most software developers like writing functional code but don't like writing test code.  With test driven development, the test code is written in the same language as the functional code, so why is it that we enjoy writing functional code but not test code?  Isn't code code?

I found myself drawn towards the functional code, but forced myself to begin by writing:

int max3(int a, int b, int c) {

  return -1;

}

I then wrote a test harness with the sample data and ran the tests.  As expected, all of the tests failed except one case where -1 was the expected result.  Next, I updated the function to return the maximum value, re-ran the tests, and they all passed!  So far so good...

In class, the TA asked the students a teaser question - write the same function using only a single if statement. My original implementation had two if statements, so I decided to give it a try and I came up with this:

int max3 (int a, int b, int c) {

   int max = a;

   if ( (b>max) && (max=b) && (c>max)) max = c;

}

I ran the tests, and they all passed so I thought I was pretty smart.  Always curious and never satisfied, I decided to add a few more tests.  To my surprise (and disappointment), this test case failed:

a = -2, b = -2, c = -1, expect: -1

Guess I'm better at writing tests than code and I'm not as smart as I first thought.  Given a specific failing test case, it was pretty easy to find the flaw in the code so I updated the function to:

int max3 (int a, int b, int c) {

  int max = a;

  if ( ((b>max) && (max=b)) || ((c>max) && (max=c)));

  return max;

}

I re-ran the tests, and they all passed.  I'm sure a more skilled developer would have recognized the flaw in my first attempt at using a single if statement, but I'm also sure that even the best programmers make mistakes.

[Update 07-Nov-2010:  OK, this code doesn't work either (it has the same type of flaw as my first attempt) and the tests did *not* all pass.  When I ran the tests, I took a quick look at the tests that failed with my first attempt, and those cases passed.  The unit tests worked great - I just made the mistake of rushing and not checking all the results...]

When I started on this exercise, I thought it was too trivial to be of much interest and pursued it solely as a simple way to start exploring test driven development.  To my surprise, this simple example illustrated an important concept - don't write "clever code."

Attempting to be too clever obscures the purpose of the code, making it difficult to understand and maintain, and increases the odds of introducing subtle, hard to detect errors.  Imagine trying to debug the flawed max3 implementation if it was buried deep inside 1,000's of lines of code, with the flaw exposed by some complex system test.  System level tests would work correctly in many cases, but then fail for unexplained reasons in other cases, leading to a long and difficult debugging session.  With a simple failing unit test, it was quick and easy to find the flaw by inspection.

Although this example is far too simple to draw conclusions about the value of Test Driven Development for commercial software development projects, I do feel it's worth exploring the technique in more complex situations.

Back to School

My son started at RPI as a freshman majoring in Computer Engineering this past September.  Reflecting back on my time in school, I noticed that some things have changed dramatically while others are pretty much the same.  Dorms now have WiFi and cable TV, but they're still dorms, and students still sleep until noon (or later) on weekends.  One exciting difference is the use of the Internet for managing classes and course work.  All of my son's class materials are posted on-line, so I've been "going back to school," following along with some of the courses as time allows.

When I took my first computer science class at UCONN, we learned PL/I, typed programs on punch cards, and stood in line waiting to put our cards through a reader to run on some mysterious IBM mainframe that we never saw.  Some time later, a printout with your last name on it came rattling out of a line printer the size of three washing machines.  When the printer ran out of paper, the cover opened up automatically, sending any punch cards on top up in the air and fluttering to the floor completely out of order.  Many of us learned not to put our cards on top of the printer the hard way.

I'm following Computer Science I at RPI, reading all the lectures and doing the labs and homework.  The course is in C++ and students work on  laptops using either Visual C++, Cygwin, or native Linux.  I work on my laptop at home, download assignments from RPI's website, edit with emacs, compile with g++ and run on a virtual remote Linux desktop using an NX Client.  Sure beats the heck out of trudging across campus at night to the computer center to wait in lines, type punch cards, and debug programs reading one print out at a time.

So far, it's been a great way to refresh my programming skills and learn emacs, which I've wanted to do for years but have never found the time to do.  Best of all, there's no (extra) cost!