crpcut ChangeLog

2016-03-07 1.9.4
  • Fixed bug where current_test was not set in --single-shot mode, causing the reporter adapter for Trompeloeil mocking framework (https://github.com/rollbear/trompeloeil) to fail.
2015-09-21 1.9.3
  • Adaptations for C++14
  • Adaptations for std::unique_ptr<> instead of deprecated std::auto_ptr<> when so available.
  • Minor bug fixes
  • Verified support for g++4.9, g++5.1, g++5.2, clang++3.7 and clang 3.6
2013-04-08 1.9.2
  • Adapted CMakeLists.txt for CMake 2.6 or later
2013-04-06 1.9.1
  • libcrpcut_basic.so was built but not installed
2013-04-05 1.9.0
  • When compiling with c++11 support (-std=c++11 or -std=c++0x) in gcc 4.6 or later, or clang 3.1 or later, tags are no longer restricted to being valid identifiers nor do they need to be defined before use with the DEFINE_TEST_TAG() macro (which becomes a no-op.)

    The macro WITH_TEST_TAG(...) accepts anything that the preprocessor can interpret, as long as the length is 80 characters or less.

  • Added an attribute location="file:line" to the XML report entries fail, info and violation. The previous use of file:line as the first line in an entry is discontinued.
  • Added the posibility to use crpcut without the heap instrumentation by providing the library libcrpcut_basic.so. This enables tests for code that for various reasons implements its own heap functions.

    If you use the heap instrumentation functionality (crpcut::heap::set_limit(n), crpcut::heap::allocated_bytes(), crpcut::heap::allocated_objects(), ASSERT_SCOPE_HEAP_LEAK_FREE or VERIFY_SCOPE_HEAP_LEAK_FREE) you link with libcrpcut.so, as usual.

    A library libcrpcut_heap.so is also provided as an experiment.

  • garbage (binary timestamp) was printed to stdout when starting a test in --single-shot mode.
2013-03-16 1.8.4
  • Fixed a bug in the heap valgrind instrumentation which made valgrind miss a lot of errors.
  • Fixed a test-dependency bug, where tests that were not selected could prevent selected tests from running.
  • Fixed a bug where INFO in tests failed when run with --single-shot
  • Fixed a bug where the duration_us field in XML reports were wrong when run with --disable-timeouts
  • Improved robustness in heap-error reporting
2012-07-21 1.8.3
  • Added file:line as first line of all reports, except info, stderr and stdout. When an exact location for the error can't be found, for example if the test has exited unexpectedly, the location refers to the beginning of the test definition.
  • Fixed a cosmetic bug where disabled tests would show up in a test listing, even though they did not match the tag filter.
  • Added an attribute "duration_us:unsignedInt" to each test in the XML-formatted reports. The time measured is the number of microseconds (real time) from starting the fixture construction up until handling the death of the test process. Currently the duration is only shown in XML-reports.
2012-07-05 1.8.2
  • Fixed a bug where the elements in expressions in ASSERT_TRUE(), ASSERT_FALSE(), VERIFY_TRUE() and VERIFY_FALSE() had to be available by reference (and thus could not be used for class scope static const value-only members.)
  • Fixed a bug where INFO and FAIL would throw std::bad_alloc when the heap space was configured too low in a test.
  • Improved performance and robustness by simplifying communication between test and test runner.
2012-06-12 1.8.1
  • --timeout-multiplier=factor now also has effect on fixture construction and destruction timeouts.
2012-06-06 1.8.0
  • By request, a command line option --timeout-multiplier=factor, where factor is a positive integer, has been added. It extends all timeouts. This can be useful if unit tests are run by a continuous integration tool and are run in a very slow environment, like under valgrind or a code coverage tool.
  • Fixed an output buffering bug where deallocated memory could be accessed.
2012-05-17 1.7.2
  • Made the NO_CORE_FILE test modifier work correctly when on Linux /proc/sys/kernel/core_pattern is a pipe (which is the default on Ubunte 12.04 LTS.)
2012-04-06 1.7.1
  • Fixed an output buffering bug which could propagate blocked writes to output intensive tests, causing them to time out.
2012-03-11 1.7.0
  • Added a new type of predicate, inheriting from crpcut::predicate, that can be matched using a pseudo operator =~ in ASSERT_TRUE(), ASSERT_FALSE(), VERIFY_TRUE() and VERIFY_FALSE(). crpcut::regex is one such example, where it is now possible to write:

    ASSERT_TRUE(string_value =~ crpcut::regex(expr_str));

    This idea was flagrantly stolen from a tounge-in-cheek mail sent to the accu-general mailing list by Jonathan Wakely in a discussion about a very similar topic.

  • A new function template:

    void crpcut::show_value<T>(std::ostream&, const T&)

    is added to be specialized if you want to control the display of values of type T in crpcut reports.

    By default std::ostream& operator<<(std::ostream &, const T&) is used when available, or a hex-dump otherwise.

  • DISABLED_TESTs are now included in the statistics of registered tests and blocked tests. The list of blocked tests is enhanced with the importance of the blocked test. In text-mode output (default to stdout) this means that each test name is prepended by one of: - disabled test ? non-critical test ! critical test.

    In XML mode, the schema has been enhanced such that each test listed in the blocked_tests section has an additional attribute "importance" with the value being one of "disabled", "critical" or "non_critical".

    This rewrite also fixes a bug where disabled tests would run if another test stated a dependency on it.

  • Fixed a race condition where the test engine could crash after a test had failed on a VERIFY_* macro
  • There is no longer a limit on the number of tests that can be run in parallel (requested with the -c / --children command line parameter.)
  • Dropped support for ancient Intel C++ Compiler
2012-01-24 1.6.1
  • Fixed a bug where the XML-formatted output could become malformed by missing a close on <blocked_tests>
2012-01-19 1.6.0
  • Added an optional third parameter to ASSERT_THROW() and VERIFY_THROW(). The extra parameter is used to match the contents of the exception. By default C-strings and crpcut::match<crpcut::regex>() are supported, for exceptions derived from std::exception, and the what() string is matched as equal to the C-string or matching the regular expression.
  • Added two more flavours of crpcut::get_parameter().

    crpcut::get_parameter<std::istream>(name) and crpcut::get_parameter<crpcut::relaxed<std::istream> >(name).

    Both can be used to extract values, possibly using stream manipulators, from the named parameter. The difference is that the former of the two will fail the test if stream extraction fails, whereas the latter will just set the stream state for you to check.

  • Added option to build libcrpcut as either a static or shared library. Shared library is the default. To build crpcut with a static library, add the flag -DBUILD_SHARED_LIBS=false to the CMake command line.
  • Fixed a heap init bug which in some cases could cause a deadlock at start up.
  • Changed the default propagation rules for text report (colour) decoration configuration from the CRPCUT_TEXT_DECORATION environment variable. See the reference manual for details.
  • Improved error handling for faulty tags provided in CLI-flags.
2012-01-03 1.5.0
  • Added experimental support for colouring the human- readable test reports. The exact format (and name) of the environment varibale CRPCUT_TEXT_DECORATION may change.

    See the chapter "Environment variables" in the reference- manual for details.

  • Made a real fix for the librt dependency problem patched in 1.4.1. This means a slight difference in behaviour if you use the CRPCUT_WRAP_FUNC or CRPCUT_WRAP_V_FUNC. The difference is that now the functions will load a library if needed to find the symbols, whereas it earlier only looked into what was already loaded. This may also point to slightly more false leaks reported by valgrind.

    See http://bugs.kde.org/show_bug.cgi?id=233298

    It is now recommended to build test programs with the -pthreads flag to gcc/clang/icc.

  • Reduced stack-trace for heap violations to only include the frames from the heap function back to the test function.
  • It is now possible to set tags to tests. The tags can be used for selecting which tests to run (in addition to name matching, that is,) and also to define which tests are critical and which are not. By default all tests are critical. Tests witout tag are always critical.

    See the reference manual for the DEFINE_TEST_TAG(), the test modifier WITH_TEST_TAG() and the command line options -L/--list-tags and -T/--tags.

    To visualize the result, the test report summary has changed to include tags and critical/non-critical PASS/FAIL.

    In human readable output, PASSED/FAILED for each test has a '!' (critical) or a '?' (non-critical) suffix. If tags are used, a summary is listed for each tag (number of tests, number passed and number failed.) The summary adds information about critical and non-critical PASSED/FAILED counts.

    In XML output, each test has an added attribute "critical". A new list - tag_summary - is adedd, which includes each tag name, the number of passed/failed and whether it is critical or not.

2011-12-20 1.4.1
  • Fixed a link dependency problem that caused crpcut to SIGSEGV at startup unless the test program had a dependency on librt.
2011-12-09 1.4.0
  • XML formatted output now refers to the XML-schema by versioned http-link instead of unversioned file name.
  • Changed the 'phase' pinpointing in violation reports. It now indicates what phase the test was in when the violation occurred. So, a violation phase of 'creating' means that the fixture constructors had not run to completion. A phase of 'running' means that the test body has begun execution, but has not run to completion. The phase 'destroying' means that the fixture destructors have begun execution, but have not run to completion. If the phase is 'post_mortem' it means that the destructors have completed without discovering any errors, but on cleaning up after the end of life of the test process, something unexpected turned up. The phase 'child' still means that the test process has fork:ed and a child process has either returned to crpcut, or has called crpcut code (e.g. ASSERT macros.)
  • Added character set conversion from test program to output. By default the test program is assumed to produce strings in UTF-8. The new command line option -C / --output-charset is used to specify the character set to use in the presentation (only available it text representation, since the XML format always uses UTF-8.) The new command line option -I / --illegal-char is used to specify how characters that are illegal in the chosen output character set are to be presented.

    XML output defaults to representing illegal characters with &#xfffd; and text output defaults to using '.' for representation of illegal characters.

    Should the test program produce strings in another character set than UTF-8, the function crpcut::set_charset(const char* set_name) is called before running the tests.

    Character sets are named as required by iconv.

  • Added possibility to register custom exception type describers; function bodies that accepts an exception and returns a textual representation of the content of the exception. See the macro CRPCUT_DESCRIBE_EXCEPTION in the reference manual.
  • For performance reasons, the display of stack backtraces when reporting heap errors is disabled by default, even when support for it is enabled in the build. The display of stack backtraces in the heap error reports can be turned on with the new -b / --backtrace-heap command line flag.
2011-07-30 1.3.2
  • Now show stack backtrace for heap errors if support for backtrace is found by CMake. Support for this can be disabled by adding -DUSE_BACKTRACE=no on the CMake command line when building the cmake library.
  • Improved the behaviour of the heap instrumentation at startup. There are still typically 6 false leaks reported by valgrind, which I believe is a bug in valgrind that doesn't count VALGRIND_MEMPOOL_FREE(pool, addr) as free'd.
  • Reduced compile time for tests. Typical reduction is around 20%, but as much as 50% improvement has been seen, depending on compiler switches and hardware.
  • Incorporated Mattias Holmlund's html-display from https://github.com/mattiash/crpcut
2011-05-30 1.3.1
  • Added an optional ID for a test-run. This ID is provided with the -i / --identity command line parameter, which requires a string value. If an identity is supplied, id "id" tag is available in the XML crpcut-header. crpcut.xsd is updated with this optional tag.
  • Fixed a bug with the XML-string encoding which allowed illegal characters. Now all non-legal characters are converted to &#xfffd;
2011-05-21 1.3.0
  • Added an optional second parameter to EXPECT_SIGNAL_DEATH and EXPECT_EXIT, which defines an on-success action. Currently the only on-success action available is WIPE_WORKING_DIR, which removes all files in the test case working directory. This can be handy for negative tests of functionality that produces data in the working directory.
  • Verified interoperability with google-mock version 1.6.0.
  • Added timeout protection for fixture construction and destruction. By default the timeout is 1s. If more time is need, the test modifiers FIXTURE_CONSTRUCTION_DEADLINE_REALTIME_MS(ms) and FIXTURE_DESTRUCTION_DEADLINE_REALTIME_MS(ms) can be used.
2011-01-23 1.2.0
  • Added VERIFY_ macros to mirror the ASSERT_ macros. The difference is that execution of a test continues after a failed VERIFY_, although the test is marked as failed, whereas a failed ASSERT_ immediately terminates the failed test.
  • Added the 'fail' entry to the list of entries that can occur in a result log for a test. This is a change in the XML Schema.
  • Added ASSERT_SCOPE_HEAP_LEAK_FREE, making it simpler to assert that a block of code deas not leak heap objects. Failed assertions list all objects that were added in the block of code that still remain allocated at the end of the block.
  • Added experimental support for storing the stack backtrace for heap objects, displayed in failed ASSERT_SCOPE_HEAP_LEAK_FREE and in alloc/dealloc type mismatch messages. The experimental backtrace support is enabled with -DUSE_BACKTRACE=yes on the cmake command line. It requires two non-POSIX functions in libc.

    - int backtrace(void**, int)

    - char** backtrace_symbol(void**, int)

  • Added ASSERT_SCOPE_MAX_REALTIME_MS(), ASSERT_SCOPE_MIN_REALTIME_MS() and ASSERT_SCOPE_MAX_CPUTIME_MS() for asserting that blocks of code meets performance requirements.
  • Added the --disable-timeouts command line flag, which disables all deadline expectations and scoped time asserts, such that no tests will fail due to their time consumption. This turns out to be useful when running tests under time consuming tools.
  • Tightened a file descriptor leak in the main process.
2010-11-21 1.1.1
  • Improved the control over naming of the lib directory from the cmake command line. Both -DLIB_SUFFIX= and -DLIBRARY_OUTPUT_PATH= can be used. The former just controls what suffix to add to the library name as "lib{suffix}", whereas the latter defines the entire name, ignoring the LIB_SUFFIX variable value.
  • ASSERT_TRUE and ASSERT_FALSE now uses some type evaluation abuse that lets them intrepret expressions. An example failed exception report is:

    ASSERT_TRUE(n + num < 5) where: n + num < 5 is evaluated as: 4 + 3 < 5

    By language rules the C++ operator presedence is obeyed, which imposes some limits on what information you can get from the expressions depending on how you write them. For example, a minor rewrite of the above example gives:

    ASSERT_TRUE(5 > n + num) where: 5 > n + num is evaluated as: 5 > 7

    If a subexpression is of an unstreamable type a hex-dump is used if the object size is eight bytes or smaller, otherwise the value is just represented by '?'.

    GCC 4.1 is picky and requires a public copy constructor for the objects in the sub-expressions.

    Thanks to Kevlin Henney and Phil Nash for bringing this technique to my attention.

2010-10-25 1.1.0
  • It is now possible to expect realtime timeouts with the EXPECT_REALTIME_TIMEOUT_MS() macro. This can be combined with DEADLINE_CPU_MS() to effectively verify that a test sleeps.
  • Buffers output internally to prevent "tee" and pagers from blocking crpcut.
  • Removed monitoring of deadlines when run with -s (--single-shot). This mode is almost exclusively used when running a test in a debugger, and information about missed deadlines is just noise.
  • Fixed long standing bug in -s (--single-shot) which allowed a test to continue despite failed asserts.
  • Adaptations for google-mock 1.5.0 (check if pthreads is required and change a pattern matching string in selftest)
2010-04-12 1.0.3
  • Fixed an error in the valgrind instrumentation of the heap, which prevented detection of overrun and underrun errors on heap objects.
  • Call the registered new_handler (if any) when new/new[] fails, as per 18.4.2 and 18.4.2 in C++ 2003 (ISO/IEC 14882:2003.)
2010-02-22 1.0.2
  • Added long-name command line parameters for improved readability.
  • Dependencies are only in effect between tests selected on the command line (all tests, in case none are explicitly selected.) This is to avoid the need of the '-n'/'--nodeps' command line flag when running individual tests with stated dependencies.
  • Improved information when test selection from the command line doesn't match or is incompatible with command line flags.
2010-01-04 1.0.1
  • Lift arbitrary parameter number limits with the use of variadic templates, for compilers that supports it.
  • Document the parameter number limits where applicable.
  • Fix a documentation bug in the FAQ, so it recommends calling InitGoogleMock to pass command line parameters to google-mock.
2009-12-20 1.0.0
  • Small documentation changes
2009-12-15 0.8.4 (a.k.a. 1.0-RC2)
  • Improved error reporting when a test case spawns a child process which runs crpcut code.
  • Allow the use of INFO and FAIL streamers also for objects which lacks output stream operator (by printing a hex dump of the memory region.)
  • Improved readability of hex-dumps for large objects.
  • Complemented the XML version of the statistics block to include selected_test_cases and untested_test_cases (both as unsigned integers.) Since this is purely an addition, the XML-schema change is not expected to break any XSLT processing of the result.
  • Fixed an internal bug that caused a pipe to be double-closed.
2009-12-07 0.8.3 (a.k.a. 1.0-RC1)
  • The most visible update is the documentation, which received a total remake. Read online on http://crpcut.sourceforge.net, or download it separately (link on the homepage.)
  • You no longer need to access crpcut functions through the crpcut::test_case_factory singleton. Functions exist directly under namespace crpcut.
  • Fixed a bug with regular expression which wrongly reported out of memory on some patterns.
  • Fixed a bug when a test case forked a child process, and misbehaved. If the child never died, the test case became a zombie and would hold up crpcut indefinitely. If the child executed crpcut macros, the engine got confused beyond all hope of recovery.
  • Yet a namespace pollution was cleaned up.
  • Adding new descriptions to libraries when creating wrapper functions is simplified. Please see the documentation for details. Unfortunately this change breaks existing code, but it does fortunately not require much editing.
  • Minor improvements to some error reports.
2009-11-15 0.8.2 (a.k.a. 1.0-beta-3)
  • Fixed accidental namespace pollution that unfortunately prohibited a number of perfectly reasonable test case names.
  • Improved error reporting from EXPECT_EXCEPTION when the exception thrown is of the wrong type, but inherits from std::exception
  • Fixed separation if log-items in text test reports.
2009-11-04 0.8.1 (a.k.a. 1.0-beta-2)
  • Delayed hooking with Google test reporter until test case starts, which avoids a spurious SIGSEGV at exit in some circumstances.
  • Fixed bug in calloc(), which wrote to NULL when memory allocation failed.
  • Fixed error reporting when heap-failures were discovered outside of test cases.
2009-11-01 0.8.0 (a.k.a. 1.0-beta-1)
  • crpcut has entered Beta stage!
  • Google-mock support is now official instead of a dirty hack. Only google-mock 1.4.0 and later is supported. The support lib libcrpcut-gmock.so no longer exists. The CMake option -DWITH_GOOGLE_MOCK=yes now only controls whether google-mock tests shall be included in selftest. It is otherwise not needed.
  • Added heap-management, through which you can query the heap for allated objects and allocated bytes. You can also limit the available heap to test out-of-memory situations. The crpcut heap instruments valgrind (http://valgrind.org) if available, to aid search for memory leaks.
  • Build support is simplified. crpcut detects the compiler used, and adapts at compile time. It is no longer necessary to add flags like -Ddecltype=typeof or -std=c++0x.
  • Support for Intel C++ compiler ICC http://software.intel.com/en-us/intel-compilers/
  • Improved readability on the test result reports. Unfortunately this includes a slight change to the XML-schema for XML-output. See crpcut.xsd.
2009-10-19 0.7.3
  • Added support for google-mock 1.4.0
2009-10-15 0.7.2
  • Small bugfixes and reduced memory footprint
2009-10-11 0.7.0
  • Added support for tests to have dependencies to testsuites, and for testsuites to have dependencies to tests and other testsuites.
  • Added support for ULPS difference tests for floating point values.
  • Changed command line support for single tests in the main process for debugging.

    Use "-s testname" (for single shot) instead of "-c 0 -n" to run a single test case in the main process.

  • Fixed comparison between pointers and 0 in ASSERT macros.
  • Fixed comparison between pointers to members and 0 in ASSERT macros.
  • Fixed support for static const integer/enum members in ASSERT macros without forcing instantiation.
  • Fixed support for non-copyable types in ASSERT macros.
  • Fixed support for stream manipulatiors in INFO/FAIL stream macros.
  • Fixed a bug in the heap of test cases violating their maximum allowed time.
2009-04-06 0.6.2
  • Fixed ASSERT_* for non copyable objects
  • Fixed ASSERT_EQ for pointer values
  • Fixed hex dump formatting
  • Fixed printing of pointer values
  • Fixed 2728096 - Support test programs compiled with -fno-exceptions with gcc. No-exception support with other compilers is still lacking.
  • Fixed numerous warnings (some were real bugs)
  • Fixed include path bug
  • Added -DCMAKE_BUILD_TYPE={release,debug,maint}
  • Much improved CMakeLists.txt making use of --std=c++0x if available
  • Moved match_traits<> out of namespace crpcut::datatypes to namespace crpcut. Sorry for the inconvenience.
2009-03-31 0.6.1
  • Added missing includes to crpcut.hpp
2009-03-31 0.6.0
  • Fixed broken wrapping of select()
  • Improved exception catching in the ASSERT_* macros
  • Changed the make system such that the test program is not built by default
  • Added string comparisons - regex for matching regular expressions - collate for locale dependent sort order
  • Added floating point comparisons - abs_diff for values near 0 - relative_diff for large values
  • Code and documentation cleaning
2009-03-18 0.5.1
  • Fixed test case termination bug which caused gcov to malfunction
2009-03-15 0.5.0
  • Added ASSERT_PRED macro, which accepts as predicate anything that looks like a function returning a value usable as a boolean expression. Predicates can optionally define an output stream operator, which will be used in violation logs, if available.
  • Added a mechanism to send parameters to the test cases through the -p command line switch, and the get_parameter() family of static functions in test_case_factory.
  • Fixed bug in command line parsing.
2009-03-05 0.4.0
  • Test cases now have a default real-time timeout of 2000ms. This gives them a 3000ms run-time before they are killed. Change it upwards or downwards per test- case as required.
  • Updated the documentation on how to link when using crpcut with google-mock. The earlier described method no longer works.
  • Added a get_start_dir() function to test_case_factory, which can be used to query the working dir the test-program was started in. Since each test case runs in its own working dir, this information is otherwise lost once the test case creation has begun.
  • Changed the termination tag to violation, and added the attribute "phase" which may have either of the values "creating", "running", "destroying" and "post_mortem" which should make error pinpointing a bit easier.
  • Default stdout to human readable format and file output to XML. Added -x flag to invert.
  • A number of lib names are now chained when looking for a symbol to wrap. Please see the documentation for details.
  • No longer exposes the crpcut::wrapped functions - it was a major mistake. crpcut itself still wrapps all functions it needs in namespace crpcut::wrapped. For your test- cases, you wrap the functions you need in whatever namespace you please. Sorry for the inconvenience.
  • Sourceforge now hosts the GIT repo as well. git://crpcut.git.sourceforge.net/gitroot/crpcut
  • Fixed a number of bugs
2009-02-16 0.3.0
  • No XML output when running tests with -c 0 for debugging.
  • Improved documentation and source-level enforcement on how to link when using google-mock.
  • Use libc/librt functions from dlopen()/dlsym(), and wrap them in namespace crpcut.
  • Added macros for creating own function wrappers.
  • Output unstreamable expressions as hex dumps.
2009-02-08 0.2.0
  • Optional support for google-mock. See README_GOOGLE_MOCK for details.
  • Added INFO and FAIL output streamers from test cases.

    INFO << expr1 << expr2 << expr3

    Sends the streamed output to the result log, but does not change the state of the test.

    FAIL << exp1 << expr2 << expr3

    Immediately terminates the test case as failed, with the output streamed data as the error cause.

  • A number of new command line flags:

    -d flag for selecting named working directory

    -o flag to direct XML output to a named file

    -q flag for silencing summation print when -o is used

    Use -n flag instead of -d for nodeps

  • Fixed numerous problems with out-of-tree builds.
  • Improved selftest.
2009-02-03 0.1.1 Bugfix patch release
  • TR 2558190 - Install fail when building out-of-tree
2009-02-01 0.1.0 First release