SafetyNet - C++ Unit Testing Framework Guide

Web: http://safetynet.devmentor.org
Email: safetynet@devmentor.org

Intro

This guide shows the student how to hand-craft a unit test class. The simpler and quick way to create a test project is to use the utgen command-line utility. However you will still need to add new test cases by hand, so this guide is invaluable in showing you how.

utgen generates code for:

1. Unit test classes
2. Test project file
3. Makefile

SafetyNet is developed on the principle of getting things done fast and easy. To that point, the source code testing framework is made to allow the developer to easily write white/grey-box unit test cases. There are only a hand-full of macros you need to know. Most of the time your task comes down to returning a boolean 'true' or 'false' value from a unit test method.

Once you get into the habit of writing unit test code, you will always want to unit test everything you code. You will have the confidence that there is a test harness to do regression testing when changes are made. As new bugs are discovered and fixed you should be adding new test cases to cover code fixes. With this practice in place, anyone else can come in, make code changes, run the unit tests and have the confidence they have not broken existing functionality. Likewise you can go in fearlessly and refactor code and have a safety-net to catch issues early during those times when turn-around is crucial to the project.

You will discover unhandled errors sooner as you write unit test code that will get your code to fail. I make it a game to try to fail my methods and APIs. Only by doing this will one know if their code is robust. Always run your unit test before changes are committed to a source control, and before all core-reviews, make this a practice! Your boss will love you for it.

Getting Started

The following steps are for general understanding, you can skim through the reading and start unit testing your code right away! In practice you should use the utgen utility rather than do these steps by hand.

Step 1

Create a Test class and subclass the Test class from class UnitTestRunner.

// File: MyUnitTest.h

class MyUnitTest : public UnitTestRunner
{
};

Step 2

The test class needs to register with the SafetyNet framework. To do this add REGISTER_TESTCLASS( ) passing the name of the Test class as a parameter. The macro must be placed in the public scope of the class definition.

class MyUnitTest : public UnitTestRunner
{
   public:
   REGISTER_TESTCLASS( MyUnitTest )
};

Step 3

The Test class must implement interface IUnitTestRunner methods.

struct IUnitTestRunner
{

   virtual void Setup()   = 0;
   virtual void CleanUp() = 0;

   virtual void RunTest() = 0;
};

Methods Setup( ) and CleanUp( ) are optional, they provide support to create and release a test fixture when each unit test is executed.

The Test class must override method RunTest( ), it will contain the code to call each unit test.

class MyUnitTest : public UnitTestRunner
{
public:
   REGISTER_TESTCLASS( MyUnitTest )

   // optional method
   void Setup();

   // optional method
   void CleanUp();

   // required method
   void RunTest()
   {
   }
};

Step 4

Two macros that initialize and terminate the test run shown below need to be added. The macros must be placed inside method RunTest( ).

   UNIT_TEST_START();
   UNIT_TEST_END();

At this point the Test class header file should look like the one shown below.

class MyUnitTest : public UnitTestRunner
{
public:
   REGISTER_TESTCLASS( MyUnitTest )

   void Setup();
   void CleanUp();

   void RunTest()
   {
      UNIT_TEST_START();

      UNIT_TEST_END();
   }
};

Step 5

Now we need to write the unit test methods. The function signature of a test method is:

A unit test method does not take any arguments and must returns a boolean value. A value of true signifies the unit test passed, whereas a false value indicates the unit test failed.

Let's add 4 unit test methods: Load, Save, Connect, Open

class MyUnitTest : public UnitTestRunner
{
   bool Load();
   bool Save();
   bool Connect();
   bool Open();

public:
   REGISTER_TESTCLASS( MyUnitTest )

   void Setup();
   void CleanUp();

   void RunTest()
   {
      UNIT_TEST_START();
      UNIT_TEST_END();
   }
};

At this point the Test class is almost complete, what's left is to add hooks that will allow the SafetyNet framework to call each unit test method and monitor the result.

Step 6

To allow the SafetyNet framework to monitor the unit test case methods, you will need to add the UNIT_TEST() macro that takes as a parameter the name of a unit test method. This macro is to be used inside method Run( ) and placed between the UNIT_TEST_START and UNIT_TEST_END macros.

// File: MyUnitTest.h
class MyUnitTest : public UnitTestRunner
{
   bool Load();
   bool Save();
   bool Connect();
   bool Open();

public:
   REGISTER_TESTCLASS( MyUnitTest )

   void Setup();
   void CleanUp();

   void RunTest()
   {

      UNIT_TEST_START();

         UNIT_TEST(Load);
         UNIT_TEST(Save);
         UNIT_TEST(Connect);
         UNIT_TEST(Open);

      UNIT_TEST_END();
   }
};

Step 7

The final step is to create the .cpp source file for the Test class with all the unit test case methods. That file will need to include the header file for the Test class along with the DECLARE_TESTRUNNER( ) macro to register the Test class with the SafetyNet framework.

// File: MyUnitTest.cpp

// includes all required UnitTest headers your test class header file
#include <UnitTest.h>      
#include "MyUnitTest.h"


// declare unit test class
DECLARE_TESTRUNNER( MyUnitTest );

When the global instance of the Test class is created, using the macro DECLARE_TESTRUNNER( ), the Test class will be register with the SafetyNet framework.

From the test class, you will need to contain one or more instance of the classes that the test will be conducted on. I will refer to these "fixtures" as Subject classes. These Subject classes can be made member of the Test class, then each Subject class can be created inside the Setup( ) function and destroyed inside the CleanUp( ) function. Since the Setup( ) and CleanUp( ) methods are called before and after each test case is run, you get the advantage of always having fresh Subjects to unit test against.

Unit test code generation with utgen

As mentioned above, since the process of writing a Test class header and source files follows a template, this process has been automated into the utgen utility. The utgen command-line tool generates both the header and source file on your behalf so you can get started with writing the unit test cases. It also generates the test project, so you can focus on writting test cases and not having to setup and create a unit test project and test classes by hand.

To generate the example test class above, utgen utility would be call with the following arguments.

The /n switch give the Test class it's name, in this case MyUnitTest.
The /t switch declares the test methods.

Here is a list of available switches at the time of writting:

   /s [subject class]
   /n [class name]
   /o [output directory]
   /a [author name]
   /c [copyright]
   /t [test case] ...
   /project exe_name test_project_path subject_source ...

The Subject class is the class you want to run the unit test on.

utgen will be post fix the test method names with '_UT', this is done to avoid compiler error resulting from name conflicts between the subject class and the test class.

Switches '/n', '/o', '/a', '/c' are optional.

If a class name is not provided (using the /n switch), then a name will be created from the subject class name by appending "_UnitTest".

If the output directory is omitted, the local directory will contain the generated files.

To generate a CMake makefile and project main source code, use switch /project
followed by the name you want the test binary to be called and then the path to the test project. After this, a list of one or more Subject class source filenames without their path should be listed.

If you forget these options simply type utgen at the command prompt and hit [Enter], this will output the usage.

NOTE: utgen produces mock classes and mock factory classes, at the time of this writing mock class testing with SafetyNet is still in the experimental stages.

A Test Run with utgen

To see how utgen works and what the generated output files look like, type the following at the command prompt:

This command will create 2 files:

1. MyApp_UnitTest.h
2. MyApp_UnitTest.cpp

The test class will contain 4 test case methods:

• Init
• Term
• Open
• Close

Below is what the source file will look like.

/**
   This unit test header file was generated by the utgen.exe utility
   Source: MyApp_UnitTest.h
   Author:
   Date: July 14, 2007
*/
   
class MyApp_UnitTest : public UnitTestRunner
{
   MyApp* m_pSubject; // Class object that unit tests get applied too!!!

   bool Init_UT();
   bool Term_UT();
   bool Open_UT();
   bool Close_UT();

public:
   REGISTER_TESTCLASS( MyApp_UnitTest )

   void Setup();
   void CleanUp();

   void RunTest()
   {
   UNIT_TEST_START();

      UNIT_TEST( Init_UT );
      UNIT_TEST( Term_UT );
      UNIT_TEST( Open_UT );
      UNIT_TEST( Close_UT );

   UNIT_TEST_END();
   }
};

Next we look at the generated source file. Noticed each test method is designed to fail and returns a boolean false value.

The utility tries to do the right thing inside the method Setup and CleanUp, but most likely this will need to be modified, or simply commented out if you don't want a new fixture created for you on each run of a test case.

/**
   This unit test source file was generated by the utgen.exe utility

   Source: MyApp_UnitTest.cpp
   Author:
   Date: July 14, 2007
*/
   
#include <UnitTest.h>
#include "MyApp_UnitTest.h"

DECLARE_TESTRUNNER( MyApp_UnitTest );

void MyApp_UnitTest::Setup()
{

   // TODO: Add code to perform initialization before each unit test is executed.
   //       Allocate all the resources here to allow the unit test to execute
   m_pSubject = new MyApp(); // TODO: use proper class constructor

}

void MyApp_UnitTest::CleanUp()
{
   // TODO: Add code to clean up after each unit test.
   //       Release all resource allocated in method SetUp()

   delete m_pSubject;
   m_pSubject = 0;
}

bool MyApp_UnitTest::Init_UT()
{
   bool bRet = false;

   // TODO: add testing code here, return true for pass, false for fail.
   return bRet;
}

bool MyApp_UnitTest::Term_UT()
{
   bool bRet = false;

   // TODO: add testing code here, return true for pass, false for fail.
   return bRet;
}

bool MyApp_UnitTest::Open_UT()
{
   bool bRet = false;

   // TODO: add testing code here, return true for pass, false for fail.
   return bRet;
}

bool MyApp_UnitTest::Close_UT()
{
   bool bRet = false;

   // TODO: add testing code here, return true for pass, false for fail.
   return bRet;
}

You should have a good understanding on the utgen utility and how to use it to generate test class. Now let's take a look at generating a test project.

Executing Your Unit Test Classes

With the Test class written you, will need to create a test project. In the test source code, you will need to create an instance of the Test framework and call it's run method. The general outline will be like the one shown below.

#include <UnitTest.h>"

int main(int argc, char* argv[])
{

   // begin the unit test
   UnitTestAssembly::GetInstance().Run();

   // free singleton unit test framework
   UnitTestAssembly::ReleaseInstance();

   return 0;
}

Our testing project does not do much, we need to be able to view the results! To do this, let's create an instance of a logger class and register it with SafetyNet.

The logger (observer) will be notified of each unit test result, as well as user generated messages and other test events. The code below shows how to get File and Console logging to take place. The File logger must be passed the name of the log file.

int main(int argc, char* argv[])
{

   // setup file logging
   StreamLogger logger("c:\\testlog\\unit_test.log");

   // setup console logging
   ConsoleLogger display;

   UnitTestAssembly::GetInstance().RegisterObserver( logger );
   UnitTestAssembly::GetInstance().RegisterObserver( display );

   // begin the unit test
   UnitTestAssembly::GetInstance().Run();

   // free singleton unit test framework
   UnitTestAssembly::ReleaseInstance();

   return 0;
}

The above code can be greatly simplified by using the macros below.

int main(int argc, char* argv[])
{

   // setup file logging
   StreamLogger logger("c:\\testlog\\unit_test.log");

   // setup console logging
   ConsoleLogger display;

   // setup test result observers

   DECLARE_OBSERVER( logger );
   DECLARE_OBSERVER( display );

   // begin unit testing
   UnitTestManager::Start();
   return 0;
}

Instead of writing this by hand, we use utgen and type the following at the console.

You will find two files:

1. SafetyNetTester.cpp
2. CMakeLists.txt

The 2nd file instructs CMake how to generate a makefile that will build your test project. For now we will ignore it's content.

Below is what get's generated.

#include <UnitTest.h>

int main(int /*argc*/, char** /*argv[]*/)
{
   // setup console logging
   ConsoleLogger display;

   // setup test result observer
   DECLARE_OBSERVER( display );

   // begin unit testing
   // NOTE: GUI Test Monitor and file logger observer are
   //       created when we start the unit test with this macro
   START_UNIT_TEST( "UnitTest.log" );
   return 0;
}

This source file will set up:

• Console logging
• File logging
• SafetyNet UI monitor logging

We show a sample output for a test run.

-----------------------------------------
Test Runner: RefCount_UnitTest
-----------------------------------------
Unit Test Started
[passed] Test case: RefCount_UnitTest::Default_UT1()
[passed] Test case: RefCount_UnitTest::Default_UT2()
[passed] Test case: RefCount_UnitTest::Default_UT3()
...

[passed] Test case: RefCount_UnitTest::RefCount_UT1()
[passed] Test case: RefCount_UnitTest::RefCount_UT2()
[passed] Test case: RefCount_UnitTest::AddRef_UT()
[passed] Test case: RefCount_UnitTest::Release_UT1()
Unit Test complete.

Test Summary: Tests(15) Fails(0) Exceptions(0)
...

----------------
Overall Summary
----------------
Total Run: 170
Total Failed: 2
Total Exceptions: 0

Recommended Reading

Once you have gained a good feel for what it takes to write a unit test, I would suggest doing the Workshop exercise to gain a mastry of SafetyNet hack-fu skills!

You can watch the videos and read the Workshop Exercise for non CMake projects write up. The utgen reference details the usage syntax.