the Compartmented Robust Posix C++ Unit Test system

Checking correctness

A test isn't worth much if it isn't possible to check the correctness of the unit under test. Although you can use the standard C/C++ assert() macro, crpcut offers a wealth of its own assert macros with several benefits.

There are two families of macros used for checking results, the ASSERT macros and the VERIFY macros. Failure to meet the expectation will fail the test. The ASSERT macros will terminate the test case immediately if the condition is not met, whereas the VERIFY macros will log the violation, mark the test case as failed, and let execution continue.

Consider the C-style intrusive list class, written below:

     
     #ifndef ilist_element_hpp
     #define ilist_element_hpp
     
     
     template <typename T>
     class ilist_element
     {
     public:
       ilist_element()
         : p_prev(static_cast<T*>(this)),  p_next(static_cast<T*>(this))
       {
       }
       ~ilist_element() { unlink(); }
       void unlink()
       {
         p_next->p_prev = p_prev;  p_prev->p_prev = p_next;
         p_next = 0;               p_prev = 0;
       }
       void insert_after(T& e)
       {
         e.p_next = p_next;    e.p_prev = static_cast<T*>(this);
         p_next->p_prev = &e;  p_next = &e;
       }
       void insert_before(T& e)
       {
         e.p_prev = p_prev;  e.p_next = this;
         p_prev->p_next = &e; p_prev = &e;
       }
       T *next() { return p_next; }
       T *prev() { return p_prev; }
       bool is_empty() const { return p_next == p_prev && p_next == this; }
     private:
       T *p_prev;
       T *p_next;
     };
     
     
     #endif // ilist_element_hpp

A few initial tests:


     #include <crpcut.hpp>
     #include "ilist_element.hpp"
     
     struct elem : public ilist_element<elem>
     {
       elem(int i) : n(i) {}
       int n;
     };
     
     TEST(create_and_destroy_empty)
     {
       ilist_element<elem> list;
       ASSERT_TRUE(list.is_empty());
     }
     
     TEST(insert_and_traverse_one_element)
     {
       ilist_element<elem> list;
       elem obj(1);
       list.insert_after(obj);
       ASSERT_FALSE(list.is_empty());
       ASSERT_TRUE(list.next()->n == 1);
     }
     
     TEST(several_elements)
     {
       ilist_element<elem> list;
     
       elem obj1(1);
       list.insert_after(obj1);
       elem *p2 = new elem(2);
       list.insert_after(*p2);
       elem obj3(3);
       list.insert_after(obj3);
     
       elem *i = list.next();
       VERIFY_EQ(i->n, 3);
       VERIFY_EQ((i=i->next())->n, 2);
       VERIFY_EQ((i=i->next())->n, 1);
       ASSERT_EQ(i, &list);
       delete p2;
       VERIFY_EQ((i=i->prev())->n, 1);
       VERIFY_EQ((i=i->prev())->n, 3);
       VERIFY_EQ(i->prev(), &list);
     }
     
     TEST(unlink)
     {
       ilist_element<elem> list;
       elem obj1(1);
       list.insert_after(obj1);
       elem obj2(2);
       list.insert_after(obj2);
       elem obj3(3);
       list.insert_after(obj3);
       obj2.unlink();
       elem *i = list.prev();
       VERIFY_EQ(i->n, 1);
       i = i->prev();
       VERIFY_EQ(i->n, 3);
     }
     
     int main(int argc, char *argv[])
     {
       return crpcut::run(argc, argv);
     }

Expressions with side effects are OK in crpcut ASSERT_*() and VERIFY_*() macros. They are carefully designed to evaluate each parameter only once.

The result of running this test program is:


     FAILED!: several_elements
     phase="running"  --------------------------------------------------------------
     samples/itest.cpp:66
     ASSERT_EQ(i, &list)
       where i = 0x7ffd6a569980
             &list = 0x7ffd6a569970
     -------------------------------------------------------------------------------
     ===============================================================================
     FAILED!: unlink
     /tmp/crpcutwiWNUr/unlink is not empty!
     phase="running"  --------------------------------------------------------------
     samples/itest.cpp:73
     Died with core dump
     -------------------------------------------------------------------------------
     ===============================================================================
     Files remain under /tmp/crpcutwiWNUr
     4 test cases selected
     
                    Sum   Critical   Non-critical
     PASSED   :       2          2              0
     FAILED   :       2          2              0

In several_elements a check fails. If the parameters to a check are output streamable, the values are printed in the report, otherwise crpcut displays a hex-dump of the objects memory. When an assertion fails, the test process is terminated. Is it the test or the list implementation that is wrong?

Another way to fail a test is to use the FAIL macro. It is used like the INFO macro described in “Test basics” on the previous page, but instead of just printing the message, the test is terminated as a failure.

The second error, in unlink, is more interesting. The expected way to leave the test function body defaults to a return (or run off the end.) Leaving by exception or, as in this case, by crashing, is failing. Since each test runs in its own process and its own working directory, the crash is not fatal to the crpcut execution, and the information in the core dump can be used when debugging the code. Can you spot the bug?

The full list of ASSERT_*() and VERIFY_*() macros for logical correctness:

macros

ASSERT_EQ(a, b), VERIFY_EQ(a, b)

Require equality between two expressions.

ASSERT_FALSE(expr), VERIFY_FALSE(expr)

Require non-truth value of an expression.

ASSERT_GE(a, b), VERIFY_GE(a, b)

Require greater-than or equal relation between two expressions.

ASSERT_GT(a, b), VERIFY_GT(a, b)

Require greater-than relation between two expressions.

ASSERT_LE(a, b), VERIFY_LE(a, b)

Require less-than or equal relation between two expressions.

ASSERT_LT(a, b), VERIFY_LT(a, b)

Require less-than relation between two expressions.

ASSERT_NE(a, b), VERIFY_NE(a, b)

Require not-equal relation between two expressions.

ASSERT_NO_THROW(expr), VERIFY_NO_THROW(expr)

Require that an expression/statement does not throw any exception.

ASSERT_PRED(pred, ...) , VERIFY_PRED(pred, ...)

Require that a predicate evaluates to true. A predicate is something that looks like a function and returns a boolean. For example:


    ASSERT_PRED(std::is_sorted<std::vector<type>::iterator>,
                v.begin(), v.end());

[Note]Note
If you use GCC version 4.3 or higher, and compile your test sources with -std=c++0x, there is no limit to the number of parameters, otherwise a predicate cannot have more than 9 parameters.

[Tip]Tip
If you use GCC version 4.5.0 or higher, and compile your test sources with -std=c++0x, you can use lambda expressions with ASSERT_PRED(pred, ...) .

ASSERT_THROW(expr, exc_type, matcher?), VERIFY_THROW(expr, exc_type, matcher?)

Require that an expression/statement throws the correct exception.

ASSERT_TRUE(expr), VERIFY_TRUE(expr)

Require truth value of an expression.