the Compartmented Robust Posix C++ Unit Test system | hosted by |
---|
1.1. | Why was crpcut written? | |||
Because, even though there are many C++ unit test systems out there, none covered all the desired properties, although many covered some:
| ||||
1.2. | Why run tests individually in separate processes? | |||
Because it protects the tests from each other. One failed test cannot easily corrupt the environment for any other test. It also makes it possible to handle inadvertent crashes and even infinite loops as any other failure. | ||||
1.3. | But doesn't forking all these short lived processes consume a lot of time, making crpcut very slow? | |||
Not really. The overhead is very low, typically much less than a millisecond, so you would need a very long suite of very short tests for this to matter. Experience shows that build-time for a test program is typically much longer than the run-time, and the test programs are usually build-once/run-once. In addition, crpcut allows you to run tests in parallel, which on a multi-core CPU can reduce run time considerably, especially if individual tests are time consuming. | ||||
1.4. | Why not throw exception to report errors? After all, that is the natural C++ error reporting mechanism. | |||
It is a bad idea just because it is the natural C++ error reporting mechanism. A unit under test could accidentally prevent an error report from being captured. Consider the following class, that needs testing: class catchall { public: catchall(int n); }; inline catchall::catchall(int n) { try { stubbed_function(n); } catch (...) { std::cerr << "something bad happened, but I saved the day" << std::endl; } } Assume a macro int expected_value = 0; void stubbed_function(int n) { THROW_ON_ERROR(n != expected_value); } Now we write a test for this program: TEST(construct_catchall_with_wrong_value) { expected_value = 3; catchall object(2); } See what's wrong? | ||||
1.5. | Why terminate a test process on the first found error? Isn't it better to continue and see if there are more errors? | |||
The thought is tempting, but if an early assertion is found to be false, how likely is the result from the rest of the code to be of any real value? In the worst case it will do something harmful. If the test is terminated immediately, it cannot do any further damage. | ||||
1.6. | Why doesn't crpcut support parametrized tests? | |||
Parametrized tests, the way they are implemented in other unit test systems, are convenient, but also makes it a bit difficult to find what was in error and what wasn't. The crpcut way of doing parametrized tests, is to express more or less the entire test functionality in a fixture using templates, and add several tests, each using different parameters for the test. Example: class parameter_base { protected: template <typename T1, typename T2> void my_test(T1 t1, T2 t2) { ASSERT_GT(t1, t2); } }; TEST(gt_int_4_int_3, parameter_base) { my_test(4, 3); } TEST(gt_double_pi_int_3, parameter_base) { my_test(3.141592, 3); }
| ||||
1.7. | How do I share a test setup that is expensive to initialize? I don't want to do it over for every test. | |||
You initialize your setup once, outside the tests. It is probably
a good idea to use a singleton, which you initialize from the
| ||||
1.8. | How do I pass state from one test to another, to chain them, if they run in isolated processes, or even in parallel? | |||
You really don't want to do that for a software unit test. If you were running a hardware production test, it might be interesting, in order to shorten the time to find defective units, but for a software unit test you want to pinpoint logical errors, and fix bugs. To do that effectively, you want to be able to run each test case individually. | ||||
1.9. | But I really really really want to chain the result of tests. How do I pass state from one test to another? | |||
Sigh, if you insist on making like difficult for yourself,
then so be it. What you do is that you set up a shared memory
region from the | ||||
1.10. | Why the macros and template trickery, instead of a more normal C++ use? | |||
It was not an easy decision, but the reason is simple. As ugly as the implementation is, the ease of use is phenomenal. You focus on writing test logic instead of writing boiler plate code to match the test engine's implementation. | ||||
1.11. | Why require functionality that isn't even standardized yet, like variadic macros and decltype? | |||
Most compilers today already support the functionality, so it isn't too outrageous. The advantage gained in ease of writing test cases far outweighs the disadvantage.
|