1.1. |
Why was crpcut written?
|
|
Because, even though there are many C++ unit test systems out
there, none covered all the desired properties, although many
covered some:
Ease of writing tests. If it isn't easy, it won't be done,
and most certainly won't be maintained.
Handle inadvertent crashes and non-progress (like infinite
wait and infinite loops.)
Make it possible to fake system errors, such as a
socket disconnected unexpectedly or disk full. The handling
of these error situations are what defines program robustness,
and they are extremely difficult to test without support from
the test tool.
Can express dependencies between tests.
|
1.2. | Why run tests individually in separate processes? |
| Because it protects the tests from each other. One failed
test cannot easily corrupt the environment for any other test. It also makes it possible to handle inadvertent crashes and
even infinite loops as any other failure. |
1.3. | But doesn't forking all these short lived
processes consume a lot of time, making crpcut very
slow? |
| Not really. The overhead is very low, typically much less
than a millisecond, so you would need a very long suite of very
short tests for this to matter. Experience shows that
build-time for a test program is typically much longer than the
run-time, and the test programs are usually build-once/run-once.
In addition, crpcut allows you to run tests in parallel, which
on a multi-core CPU can reduce run time considerably, especially
if individual tests are time consuming. |
1.4. | Why not throw exception to
report errors? After all, that is the natural C++ error reporting
mechanism. |
|
It is a bad idea just because it is the natural C++ error reporting
mechanism. A unit under test could accidentally prevent an error
report from being captured. Consider the following class, that
needs testing:
class catchall
{
public:
catchall(int n);
};
inline catchall::catchall(int n)
{
try {
stubbed_function(n);
}
catch (...)
{
std::cerr << "something bad happened, but I saved the day" << std::endl;
}
}
Assume a macro THROW_ON_ERROR
that behaves like an assert, except that it throws. It is used
by the stub to verify that the caller does the right thing.
int expected_value = 0;
void stubbed_function(int n)
{
THROW_ON_ERROR(n != expected_value);
}
Now we write a test for this program:
TEST(construct_catchall_with_wrong_value)
{
expected_value = 3;
catchall object(2);
}
See what's wrong?
|
1.5. | Why terminate a test process on the first found error? Isn't it
better to continue and see if there are more errors? |
| The thought is tempting, but if an early assertion is found
to be false, how likely is the result from the rest of the code to
be of any real value? In the worst case it will do something harmful.
If the test is terminated immediately, it cannot do any further
damage. |
1.6. | Why the macros and template trickery,
instead of a more normal C++ use? |
| It was not an easy decision, but the reason is simple. As ugly
as the implementation is, the ease of use is phenomenal. You focus
on writing test logic instead of writing boiler plate code to
match the test engine's implementation. |
1.7. | Why require functionality that isn't even standardized yet, like
variadic macros and decltype? |
| Most compilers today already support the functionality, so
it isn't too outrageous. The advantage gained in ease of writing
test cases far outweighs the disadvantage. ![[Note]](images/note.png) | Note |
---|
It is only the test code itself that must be
compiled with support for the non-standard functionality. The
units under test do not.
|
|