Nothing makes a user more annoyed then finding a bug in a otherwise useful program. Find and kill those nasty little bugs before they kill you. The only proven way to do this is to unit test your Java code early and often. You think having it print out the random debugging message is Java unit testing?!?!? The only way to prove your code works is to systematically unit test it. If you do not automate the Java unit testing process we guarantee you will not use Java unit testing. Guess who suffers? Yes, the user, but also you when they get fed up with your latest "one more bug to fix" (they would probably believe "the dog ate it" as being more likely).

We can go on forever or just show a simple example of Java Unit Testing. Find the bug in the following Java code, and find a way to make sure it will never come back once fixed:

	for(int i=0;i<=Integer.MAX_VALUE+1;i++)
		System.out.println(i);

Hint: How many times does the loop run?

Yes we admit this a trivial example but let's say that the loop termination condition is based on variable whose value just happens to be Integer.MAX_INT currently, this is not so trivial anymore. Now let's make it worse and say that the println is replaced with a call to a method that has non-linear output for n and n+1. Sorry, times up!!! The user wants to know why their bank balance is -$999,999,999.99 and will sue/fire you if not fixed in a week.

This all could of been avoided by just testing the value of i after the loop terminates. If it is not 0 (if you think it should be Integer.MAX_VALUE+1, -1 or any other value check your API documentation!) then the code is buggy by definition. Using Java 1.4+'s native facilities we could rewrite the code to read:

	int i;

	for(i=0;i<=Integer.MAX_VALUE+1;i++)
		;

	assert(i!=0);

In theory Sun's inclusion of assert's in Java should be sufficient to perform rudimentery Java unit testing but it has one major drawback:

If an assert argument is false then the program just terminates with a cryptic message that gives no clue to how the assert failed.

Also assert's embedded in production code force it to run slower then it would without them. Besides their only purpose is to crash the program if you did your job wrong... what message does this send to the user?

Separating your Java unit test code out from the production code is, for sure, a step in the right direction but it is not enough to fully unit test your Java code. We still have no way of knowing why the Java unit test code failed, just that it did. Even worse without designing for Java unit testability you might not find the bug. For example when I was still primarily a C++ programmer I wrote the following gem:

	public void *getAddr(int size)
	{
		return malloc(sizeof(size));
	}

For those of you don't know, int's are always 4 bytes in C/C++. The result? all objects were allocated 4 bytes of RAM! Since this gem was very deeply nested at the bottom of a complex memory management scheme that was invoked every time we did something like new Foo() all kinds of truly odd behavior resulted. My favorite one involved this pseudo-code:

	Fruit fruit=new Fruit(apple)

	print fruit

	if fruit==apple 
		fruit=orange

It printed "orange" every time. I spent a good week attempting to figure out how this could possibly fail. The answer was we had two values side by side in RAM since the compiler thought Fruit used, if I remember right, 64 bytes plus some pointers into the heap. So far so good but the OS only actually allocates 4 bytes regardless of the requested allocation. This means if we have two fruits on the heap then they will actually overlap and garbage the values of both.

The solution would have been to write a little test program to verify that getAddr returned a heap chunk that was actually size bytes large. For example this would suffice:

main()
{
	int *p1=getAddr(10);

	if(p2-p1!=10)
		cout<<"getAddr failed to allocate 10 bytes it only gave me "<<(p2-p1)<< endl;
}

This is OK if it fails this one test but if it doesn't print anything how do we know if it passed or just crashed. One solution is to put a whole bunch of debugging prints into your code and wade through the output looking for the rusty nail in the haystack. Not only is this tedious but it violates one of the main precepts of well-behaved programs; if you have nothing to say then say nothing.

Yes we admit that even for examples as simple as the ones we have looked at so far this approach is overkill. While it is impossible to gain the magic of Java unit testing completely painlessly it can be done with much less pain then the above examples. The secret is to use a single unified set of Java unit testing procedures, the core of which should be designed to answer one question:

Does the real output match the expected output?

If it does then out Java unit test passes, if it doesn't it fails. Repeat 10,000 times and we just proved our brand new OS is better then Windows. Not really! but it is definitely better then the ad-hoc nature of our examples so far. Besides how do we know if the output actually matches? It does give us an interesting question: what does "matching" mean? It normally means, once we get rid of all the unneeded support code, does some expression evaluate to be true or false, or that the actual result is the expected result. Case in point in the for loop example all I care about is the output (i) actual -1 or in the C++ example if I want to know if I got 10 bytes back from the malloc. The Java unit test code can be as simple as:

	assertTrue(i==-1);		// is i equal to -1
	assertEquals(p2-p1,10);		// is there a 10 byte gap between sequenciel allocations

If assertTrue and assertEquals do not have any way of reporting their Java unit test results this is even worse then no testing at all. But too much output will confuse the human tester. The solution is to give a summary of results for a whole collection of Java unit tests. Yes 10,000 Java unit tests will prove if our OS does work, but 10,000 lines of output is insane. Wouldn't this be better?

Passed: 10000
Failed: 0
No result: 0

How can a Java unit test have no result?!?!?!? Sometimes a piece of code can fail in a way that has nothing to do with the code under Java unit test and/or is repeatable. If I am testing a web server and my Java unit test programs test it by emulating a client then we have just assumed the protocol stack and/or the transport network are functioning. If neither is true then it is not possible to actually Java unit test the web server, thus the Java unit test has no result.

If one of the Java unit tests does not pass then we will want to know the critical facts about the failure and nothing more. Something like this would suffice:

	-1 != 0		TestMyClass.java:10

Namely the assertEquals on line 10 of TestMyClass failed because the actual value (-1) does not equal the expected value (0).

As they stand I wouldn't use them either! The reason being that this primative Java unit testing process is too complex to handle as a part my everyday coding tasks. Thus any Java unit testing solution needs to be:

• Automated (including verification of results via scripts)
• Integratable into my development environment
• Keep out, of my way unless something fails and then sound all the alarms

Yes we are slightly insane in suggesting that you need to write 10,000 Java unit tests to test a new OS. But, there is not a single Java unit test that upon reflection can be removed. Just as an OS, or any other large program is modular, Java unit testing can also be modular. At FloSoft Systems we are in fact in the early phases of writing an OS. While many aspects are not formalized we do know that if it succeeds it will be radically different then any other OS. This means we will be using a blend of tried and true computer science and software engineering principles and completely new ones. We are equally convinced that we have to Java unit test every last line of code. So yes we might very well have 10,000 (or more) Java unit tests when we are done.

But we need not run all 10,000 to see if our OS actual works. In this case only 6 Java unit tests are needed and their success/failure does not effect the success/failure of any other component. This means as long we can create groupings of Java unit tests that are selected at run-time it doesn't matter what tests we run. Just to prove the point here is the the entire Java unit test code for the bottom layer of our Java unit testin system:

public class TestAll extends TestSuite
{
	public void run()
	{
		addTest(new TestResult());
		addTest(new TestTest());
		// no need for a TestTestSuite or TestTestSet since they by definition work if we reach
		// this comment

		super.run();
	}
}

public class TestResult extends TestSet
{
	@TestCase
	public void testCtor()
	{
		Result res=new Result();

		assertEquals(res.getPass(),0);
		assertEquals(res.getFail().size(),0);
		assertEquals(res.getNoResult().size(),0);
	}

	@TestCase
	public void testAddPass()
	{
		Result res=new Result();

		res.addPass();
		assertEquals(res.getPass(),1);
	}

	@TestCase
	public void testAddFail()
	{
		Result res=new Result();

		res.addFail(new TestException());
		assertEquals(res.getFail().size(),1);
	}

	@TestCase
	public void testAddNoResult()
	{
		Result res=new Result();

		res.addNoResult(new Throwable());
		assertEquals(res.getNoResult().size(),1);
	}

	@TestCase
	public void testMerge()
	{
		Result r1=new Result();
		Result r2=new Result();
		Result r3=new Result();
		Result rOut=new Result();

		r1.addPass();
		r2.addFail(new TestException());
		r3.addNoResult(new TestException());

		rOut.merge(r1);
		rOut.merge(r2);
		rOut.merge(r3);

		assertEquals(rOut.getPass(),1);
		assertEquals(rOut.getFail().size(),1);
		assertEquals(rOut.getNoResult().size(),1);
	}
}

public class TestTest extends TestSet
{
	@TestCase
	public void testCtor()
	{
		Result res=getResult();

		assertEquals(res.getPass(),0);
		assertEquals(res.getFail().size(),0);
		assertEquals(res.getNoResult().size(),0);
	}
}

If you had all the time in the world then making your own Java unit testing solution is most likely the best alternative for Java unit testing your code. But you don't have all the time in the world. For that reason we have designed thisTest, the Java unit testing framework we have been discussing up to now, have the following key design goals:

1. Work with you not against you (or your enviroment)
2. Keep simple things simple and leave the complexities of how to Java unit test a project to the user (while we know how to Java unit test your software we don't know what to Java unit test!)
3. Be developer, not business, friendly. (Our code and API are 100% open for user modification)
4. Make no assumptions about the type of development enviroment it is in beyond the presence of an 1.5.0 or later JRE.

As we said from the start you will select a Java unit testing solution on techinical merrits alone and since we don't know what your exact project requirements are we offer a 30 day trial of thisTest version that is identical in every aspect to the paid version of thisTest. In other words we trust you to treat us as fellow programmers and pay for it if you find thisTest useful (it will not even stop functioning after 30 days). Our Miai Certified licensing terms forbid us from treating you any other way. In keeping with our mutual professional respect we have made the post trial period price as painless as possible and still put food on our tables ($25/executable copy). Our licensing terms offer several other ways of meeting these obligations if your are unable and/or feel wrong about paying for community-oriented software. Who knows? You could even make money from us for your use of thisTest (see license attachment A for details).