If you're testing a class that contains references to other classes that are a parts of your project then, unless you are using some sort of mock objects, it is almost 100% certain that you are not making an honest test of the class, and not the entire system. It is the equivalent of having to test every component in a computer just to test the keyboard.

For the above reason we generally divide tests into two very broad catagories:

Unit Testing

Testing each component in as much isolation as possible

Integration (aka System) Testing

Testing how components interact (this presupposes that the guts of the components are correct and thus need no testing)

For reasons explained below it is impossible to do true unit testing, in the general case, if production code is not modified. The nature of this modification will be discussed below also.

Note Just to clarify our terminology in this section "interface" refers to how two components interact and not to a interface class.

As usaual, I will illustrate my point via an example instead of attempting to show its theoretical grounding. Let's take something like a mouse and look at how we would test the hardware and the driver. The first thing to observe is as long as the driver thinks there is a "real" mouse attached to the I/O mechanism it uses to communicate with the mouse, it does not matter in the least if the thing providing the I/O is really a mouse or just a software simulation of one. The second observation is that due to the concept of functional layering a real (not simulated) mouse doesn't know or care about the "how" of how it gets external input and/or where it sends it output.

Given these two insights we see that the following functionality will have to be handled as a combination of the driver and hardware I/O:

• Position (X, Y)
• Get click count
• Last click button
• Reset click count
• Scroll Wheel movement

As would happen in real life let's assume that one team makes and tests the hardware and another makes and tests the driver at the same time. This means that there is functionality that might be implemented by one team before the other team has even started on the particular feature. For the rest of this section we will assume that the driver team has implemented the positional code but the hardware team has not even selected the type of pointer (inverted trackball, trackball, optical, graphics tablet, etc.), but they have specified what the hardware interface will look like to the host computer. They have provided the following minimum specifications:

• When moved the mouse reports its new position as a delta to the last postilion
• All position reports will be made in screen pixels (rounded off)
• Position reports are updated every 1ms
• The buffer can hold only one position at a time

First of all we should define a class to store an abstract ordered pair (x,y):

	public class Position
	{
		public Position(int x, int y)
		{
			this.x=x;
			this.y=y;
		}

		public int getX()
		{
			return x;
		}

		public int getY()
		{
			return y;
		}

		public void setX(int deltaX)
		{
			x+=deltaX;
		}

		public void setY(int deltaY)
		{
			y+=deltaY;
		}

		private int x;
		private int y;
	}

For the sake of simplicity let's say the mouse driver class is as follows:

	public class Mouse implements Runnable
	{
		public void Mouse()
		{
			hw=new MouseHardware();		// assume this the real mouse
			pos=new Position(400,300);	// center of a 800x600 screen

			// detach the driver from the caller
			Thread thread = new Thread(this);
			thread.run();
		}

		public void run()
		{
			while(true) {
				Thread.sleep(1);	// for the sake of simplicity we assume that
							// the machine this runs under can run all threads
							// on their own dedicated processor (i.e. if 
							// some calling thread sleeps for 1 ms at exactly
							// the same time as us then both threads will
							// wake exactly at the same time)

				updatePosition();
			}
		}

		private updatePosition()
		{
			int deltaX=hw.getX();
			int deltaY=hw.getY();

			pos.setX(deltaX);
			pos.setY(deltaY);
		}

		public Position getPosition()
		{
			return pos;
		}

		private Position pos;
		private MouseHardware hw;
	}

Assuming we have some test apparatus that can move the real mouse pointer in any pre-programmed "path" then all we need to do to test the mouse is program the test apparatus to say move the pointer +1 pixel in both x and y per I/O probe time (1 ms). Then the following test code will suffice to test, using FloSoft System's thisTest testing framework, the complete system (driver, Position, and the hardware):

public class TestMouse
{
	@TestCase
	public void testPointerPosition()
	{
		MouseHardwareTester.setPath("delta",1,1);	// move the mouse 1 x and 1 y exery 1 ms
		Mouse mouse=new Mouse();
		Position currPos=mouse.getPosition();

		// test for 100 ms
		for(int i=0;i<10;i++) {
			Position newPos=mouse.getPosition();

			assertEquals(currPos().getX()+1,newPos.getX());
			assertEquals(currPos().getY()+1,newPos.getY());

			currPos=newPos;
			Thread.sleep(1);
		}
	}
}

This is all well and good if the hardware team has finished the pointer hardware, which they have not, but we still need to test the driver as if it has. It is important to note for this test all we care about is does the driver track motion properly not wither it tracks the actual mouse correctly. Thus we can write a mouse similator that is something like this:

public class MouseSim extends MouseHardware
{
	public int getX() { return 1; }
	public int getY() { return 1; }
}

If at run time we substitute MouseSim for MouseHardware in the mouse instance in the driver then we need not worry if the hardware is working properly. In other words as far as testPointerPosition is concerned it is testing the real hardware.

Even though it is a little long this example illustrates the difference between testing the interface (MouseHardware.getX() and MouseHardware.getY()) and testing if the driver updates the postion correctly. The reason for all this is that the hardware team has not finished MouseHardware.get???(). The deeper (and more important) reason, at least the surface one, is in the real driver updatePosition() would likelly be considerably more complex then just calling mouse.get???() and if there is a bug in it, it would be pretty much near impossible (in the general case) to tell if the bug was caused by Mouse or MouseHardware.

There are two problems with the above style of creating a simulated mouse to be used when testing the core of Mouse, i.e. unit testing it. The first is we would need to have two classes of Mouse, like Mouse and FakeMouse, with Mouse using MouseHardware and FakeMouse using SimMouse. As long as we never make any changes to either class or MouseHardware we are pretty safe in doing this. In real life this is never the case and humans being humans, I guarantee at some point you will update the code in one but forget to update the other. The other issue is even if we keep the two versions in sync there may be subtle timing issues, especially in a device driver, that SimMouse may not exhibit (if we are not testing for these timing issues then we can safely ignore them). This means that incorrect modifications to Mouse could, in theory make it fail if SimMouse instead of MouseHardware is used. For both these reasons the swap out between real and "mock" (like SimMouse) objects should occur as late as possible in the testing, ideally at run time.

Assuming for the minute there is some magical way to replace real field instances with mock ones automatically there would, by definition, need to be a naming convention to make sure that the correct swap is made. Also we would not want to mock "unmockable" classes, thus that are primitives or members of Sun's stock Java API. FloSoft Systems's aMock is designed specifically to make this swap using such a naming convention. Refer to it for operational details on its use.

Using aMock we can rewrite the above test code as follows (assuming we make MouseHardware, Position and SimMouse all follow aMock's naming conventions):

public class TestMouse
{
	@TestCase(TestType.Unit)
	public void ubitTestPointerPosition()
	{
		Mouse mouse=new mouse;
		ClassUtils.mocify(mouse);

		_testPointerPosition(mouse);
	}

	@TestCase(TestType.Unit)
	public void ubitTestPointerPosition()
	{
		Mouse mouse=new mouse;

		_testPointerPosition(mouse);
	}

	public void _testPointerPosition(Mouse mouse)
	{
		MouseHardwareTester.setPath("delta",1,1);	// move the mouse 1 x and 1 y every 1 ms
		Position currPos=mouse.getPosition();

		// test for 100 ms
		for(int i=0;i<10;i++) {
			Position newPos=mouse.getPosition();

			assertEquals(currPos().getX()+1,newPos.getX());
			assertEquals(currPos().getY()+1,newPos.getY());

			currPos=newPos;
			Thread.sleep(1);
		}
	}
}

By calling ClassUtils.mockify(Object o) we replace all mockable fields in o with mock instances.

As mentioned last week the exact definition for unit and integration testing are largely made on a shop by shop basis. For example at FloSoft Systems we define a unit test as one where all class(es) under test are ClassUtil.mockify()'ed and an integration test is anything else. Due to having no steadfast rules and following our practice of our tools staying out of your way we have introduced the TestType parameter to the @TestCase annotation. It can have the following values:

Value	Meaning
All	"Always" run the test (default value)
Unit	It is a unit test
Integration	It is an integration test

thisTest (1.1) was enhanced at the same time aMock was released to incorporate TestType. There are two TestType values that you need to be aware of the global one and the per @TestCase one. The following table shows how thisTest decides to run a test or not:

Note: X means thisTest will run the test. Vertical axis is globals and horizontal is @TestCase value

	@TestType
	All	Unit	Integration
All	X	X	X
Unit	X	X
Integration	X		X

In closing, the way to read the new TestMouse test code is to know that since testUnitPointerPosition is marked as TestType.Unit it will be run only if the global type is All or Unit. For the exact details of how to set the global type see the documentation (errata) for thisTest.