the Compartmented Robust Posix C++ Unit Test system

TEST(name, ...)

Define a test

Used in: Global or testsuite scope. See TESTSUITE(name, ...)

A test has a name and a function body. It can optionally have fixtures and/or modifiers.

Normally a test will pass if it returns from the function body without leaving any files in the working directory and without having explicitly failed (through any of the macros: ASSERT_EQ(a, b), ASSERT_NE(a, b), ASSERT_GE(a, b), ASSERT_GT(a, b), ASSERT_LE(a, b), ASSERT_LT(a, b), ASSERT_FALSE(expr), ASSERT_TRUE(expr), ASSERT_PRED(pred, ...) , ASSERT_SCOPE_HEAP_LEAK_FREE, ASSERT_SCOPE_MAX_CPUTIME_MS(ms), ASSERT_SCOPE_MAX_REALTIME_MS(ms), ASSERT_SCOPE_MIN_REALTIME_MS(ms), FAIL). VERIFY_EQ(a, b), VERIFY_NE(a, b), VERIFY_GE(a, b), VERIFY_GT(a, b), VERIFY_LE(a, b), VERIFY_LT(a, b), VERIFY_FALSE(expr), VERIFY_TRUE(expr), VERIFY_PRED(pred, ...), VERIFY_SCOPE_HEAP_LEAK_FREE, VERIFY_SCOPE_MAX_CPUTIME_MS(ms), VERIFY_SCOPE_MAX_REALTIME_MS(ms), VERIFY_SCOPE_MIN_REALTIME_MS(ms), WITH_TEST_TAG(tagname)

Example: The smallest test that can succeed:


    TEST(testname)
    {
    }
      

It does not explicitly fail, nor does it leave files behind, thus the test passes.

A test that leaves files behind in the working directory is a failure.

Example: The test program

     
     #include <crpcut.hpp>
     #include <fstream>
     
     TEST(file_fail)
     {
       std::ofstream file("name");
     }
     
     int main(int argc, char *argv[])
     {
       return crpcut::run(argc, argv);
     }

        

fails because it left files behind.


     FAILED!: file_fail
     /tmp/crpcut1g8Wzx/file_fail is not empty!
     ===============================================================================
     Files remain under /tmp/crpcut1g8Wzx
     1 test cases selected
     
                    Sum   Critical   Non-critical
     FAILED   :       1          1              0

        

By default crpcut imposes a real-time limit of 2s for a test function. If the function is not finished by then, the process is killed and the test is failed.

Example: The test program

     
     #include <crpcut.hpp>
     
     TEST(infinite_loop)
     {
       for (;;)
         ;
     }
     
     int main(int argc, char *argv[])
     {
       return crpcut::run(argc, argv);
     }

        

is killed and failed:


     FAILED!: infinite_loop
     phase="running"  --------------------------------------------------------------
     samples/infinite_loop.cpp:30
     Timed out - killed
     Expected normal exit
     -------------------------------------------------------------------------------
     ===============================================================================
     1 test cases selected
     
                    Sum   Critical   Non-critical
     FAILED   :       1          1              0

        

Should you have a test that needs more time than 2s, you can raise the limit with the DEADLINE_REALTIME_MS(n) modifier. If the need is temporary due to running with time consuming tools like valgrind, you may temporarily extend the timeout limit using the command line parameter --timeout-multiplier=factor, or even turn off all timeouts with the -t / --disable-timeouts command line parameter.

It is important to understand that the name of a test is, in fact, the name of a class. The test body is the member function test()that is called by the crpcut engine. If more parameters are added to the TEST() macro, after the name of the test, those are expected to be types and will be base classes.

Fixtures

A fixture is a struct or class that implements functionality, typically a setup, that is common to many tests. The constructor creates the common setup, and the destructor tears it down.

When a fixture is included in the TEST() macro invocation, the test class inherits from the fixture, and thus gains access to the data and functionality of the fixture.

Example: two tests sharing a fixture

     
     #include <crpcut.hpp>
     #include <string>
     
     class fixture
     {
     protected:
       fixture() : str("hello world") {}
       std::string str;
     };
     
     TEST(first_test, fixture)
     {
       ASSERT_EQ(str, "hello world");
     }
     
     TEST(second_test, fixture)
     {
       ASSERT_EQ(str.length(), std::string::size_type(11));
     }
     
     int main(int argc, char *argv[])
     {
       return crpcut::run(argc, argv);
     }

        

It is possible to add a large number of fixtures at the same time, but the same rules apply as for any multiple inheritance.

Example: name disambiguation

     
     #include <crpcut.hpp>
     #include <string>
     
     class fixture_string
     {
     protected:
       fixture_string() : val("hello world") {}
       std::string val;
     };
     
     class fixture_int
     {
     protected:
       fixture_int() : val(11) {}
       int val;
     };
     
     TEST(first_test, fixture_string, fixture_int)
     {
       ASSERT_EQ(fixture_int::val, 11);
       ASSERT_EQ(fixture_string::val, "hello world");
     }
     
     TEST(second_test, fixture_string, fixture_int)
     {
       ASSERT_EQ(fixture_string::val.length(),
                 std::string::size_type(fixture_int::val));
     }
     
     int main(int argc, char *argv[])
     {
       return crpcut::run(argc, argv);
     }

        

A fixture is always default-constructed. crpcut does not provide any way to pass information to the constructor. More often than not, this limitation is easily overcome using templates.

Example: The test program

     #include <crpcut.hpp>
     #include <fstream>
     #include <string>
     
     extern "C"
     {
     #include <unistd.h>
     }
     
     template<const char *(&filename)>
     class file_fixture
     {
     protected:
       file_fixture()
       {
         std::ofstream of(filename);
         of << "contents\n";
         if (!of) FAIL << "Could not write data to " << filename;
       }
       ~file_fixture()
       {
         if (::unlink(filename) == 0)
           {
             FAIL << "test should've removed the file, but didn't";
           }
       }
     };
     
     const char *filename = "data.txt";
     
     TEST(read_and_remove_file, file_fixture<filename>)
     {
       std::ifstream in("data.txt");
       std::string s;
       in >> s;
       ASSERT_EQ(s, "contents");
     }
     
     int main(int argc, char *argv[])
     {
       return crpcut::run(argc, argv);
     }

        

fails because the test did not remove the file, as it should have.


     FAILED!: read_and_remove_file
     phase="destroying"  -----------------------------------------------------------
     samples/file_fix.cpp:50
     test should've removed the file, but didn't
     -------------------------------------------------------------------------------
     ===============================================================================
     1 test cases selected
     
                    Sum   Critical   Non-critical
     FAILED   :       1          1              0

        

The example above also shows that errors can be reported in the fixture constructors and destructors. The phase of the error report tells when the error was discovered.

[Note]Note

Sharing state between several tests through fixtures is possible, but it requires some planning.

Since each test is spawned off in a separate process, it is not enough to let the fixture constructor set up the shared state in global-, namespace-scope, or class-static-variables.

Instead the shared state must be set up in constructors for objects in global-, class-static-, or namespace-scope. If these objects are constructed before the test process is spawned, each test process will get a copy of it. Keep in mind that it is a copy. If you want to transfer a state change from one test to another, you have to be even more careful, using shared memory or files, and also keep in mind that the order tests are run in is undefined, except when constrained by DEPENDS_ON(...).

Modifiers

Modifiers are ways to chance crpcuts expectations and behaviour of the test. Normally, a test can be scheduled to run at any time, and for a test to pass, it must return from the test body without having triggered any error condition. However, modifiers can change that expectation.

List of modifiers

DEADLINE_CPU_MS(n)

Require that the function finishes before having consumed too much CPU-time.

DEADLINE_REALTIME_MS(n)

Require that the function finishes before having consumed too much real-time (i.e. including sleeping.)

DEPENDS_ON(...)

Define the requirements that must be fulfilled before the test is allowed to run.

EXPECT_EXCEPTION(type)

Change the success expectation such that the function must leave by throwing an exception in order to pass.

EXPECT_EXIT(num, action?)

Change the success expectation such that the function must leave through a call to the exit() function in order to pass.

EXPECT_REALTIME_TIMEOUT_MS(ms)

Change the success expectation such that the duration of the test must meet or exceed a minimum amount of time in order to pass.

EXPECT_SIGNAL_DEATH(signo, action?)

Change the success expectation such that the test process must die due to a signal in order to pass.

FIXTURE_CONSTRUCTION_DEADLINE_REALTIME_MS(ms)

Change the time allowed for the fixture construction.

FIXTURE_DESTRUCTION_DEADLINE_REALTIME_MS(ms)

Change the time allowed for the fixture destruction.

NO_CORE_FILE

Prevent the test process from dumping core.

WIPE_WORKING_DIR

Clean out remaining files after the test.

WITH_TEST_TAG(tagname)

Set a tag to a test.