the Compartmented Robust Posix C++ Unit Test system

Chapter 11. Stubbing library functions

You can safely roll your own versions of functions like fork() and clock_gettime(). crpcut make all library calls through dlopen() and dlsym(), and will not be affected by your versions[1][2]

Sometimes, however, replacing a library function is not what you want, but rather to enclose it inside a wrapper that does filtering and error injection. You can do that with the help of the macros CRPCUT_WRAP_FUNC(lib, func, rv, param_list, param_call) and CRPCUT_WRAP_V_FUNC(lib, func, rv, param_list, param_call). Here's classic difficult situation, with a process that spawns a child and interacts with it:


     extern "C"
     {
     #include <unistd.h>
     #include <sys/types.h>
     #include <sys/wait.h>
     #include <stdlib.h>
     }
     
     #include <stdexcept>
     #include <string>
     #include <cassert>
     
     class work
     {
     public:
       class fork_exception : public std::exception  { };
       class pipe_exception : public std::exception  { };
       work() :pid(-1)
       {
         int rv = pipe(fd);
         if (rv == -1) throw pipe_exception();
         rv = ::fork();
         if (rv == -1) throw fork_exception();
         if (rv == 0) { close(fd[0]); fd[0] = -1; do_work(); return; } // child
         close(fd[1]);
         fd[1] = -1;
         pid = rv;
       }
       ~work() {
         if (fd[0] != -1) close(fd[0]);
         if (fd[1] != -1) close(fd[1]);
       }
       std::string get_data() // empty data signals that child is done
       {
         size_t len = 0;
         ::read(fd[0], &len, sizeof(len));
         std::string data(len > 0 ? len : 0, '_');
         if (len > 0)
           {
             ::read(fd[0], &data[0], len);
           }
         return data;
       }
       void wait()
       {
         int status;
         int rv = ::wait(&status);
         assert(rv != 0 || status == pid);
         pid = -1;
       }
     private:
       void do_work() { /* very hard stuff */  }
       int fd[2];
       int pid;
     };

In this case, simple stubs are used to provide preprogrammed behaviour, and fall back to the normal library function when nothing preprogrammed exists.

The test program is a bit long, but take some time to read it through. For each wrapped library function, there is a configuration in the shape of a list. If the list is empty, the normal function applies, otherwise the behaviour mandated by the list-item is carried out. pipe() calls the library function, unless explicitly instructed to fail, but stores the allocated file descriptors for later use. close() always calls the library function, but if the file descriptors are allocated with pipe, they are verified to be in valid state.

[Caution]Caution
Your versions of the library functions are in effect for all tests in the entire program. Make sure they have a decent default behaviour to save yourself from debugging headaches.

A short walk-through of the tests:

fork_fails

Only one configuration which ensures that fork() fails with ENOMEM. Verify that work::fork_exception is thrown.

pipe_fails

Only one configuration which ensures that pipe() fails with EMFILE. Verify that the constructor throws work::pipe_exception

read_one_string

The complex one, where all library function wrappers are strictly controlled. fork() will return a pid, thus testing the parent process behaviour.

The pipe() wrapper lets the calls to the library function through, but it stores the allocated file descriptors for use by the read(), write() and close stubs.

The read() and write() functions are stubbed completely to verify the correct file descriptor, provide predefined data from the faked child process, and verify that the data written back is correct.

signal_on_read

Similar to read_one_string above, but the second read is set to return -1 and set errno to EINTR, to indicate that the call was interrupted by a signal.

normal_work

It doesn't set up any configurations for any of the wrapped library functions, so for this test, the normal libc implementations are run unconstrained. It will, in other words, behave as in a real program.


     #include "process-example.hpp"
     #include <crpcut.hpp>
     #include <cerrno>
     #include <list>
     #include <map>
     
     namespace original
     {
       CRPCUT_WRAP_FUNC(libc, fork, pid_t, (void), ())
       CRPCUT_WRAP_FUNC(libc, close, int, (int fd), (fd))
       CRPCUT_WRAP_FUNC(libc, pipe, int, (int fd[2]), (fd))
       CRPCUT_WRAP_FUNC(libc, wait, pid_t, (void *status), (status))
       CRPCUT_WRAP_FUNC(libc, read, ssize_t, (int fd, void *p, size_t n), (fd, p, n))
       CRPCUT_WRAP_FUNC(libc, write, ssize_t, (int fd, const void *p, size_t n), (fd, p, n))
     }
     
     //
     // data structures used to control how the wrappers behave
     //
     
     struct pipe_data
     {
       pipe_data(int retval_, int *rfd_, int *wfd_, int err_)
         : retval(retval_), rfd(rfd_), wfd(wfd_), err(err_) {}
       int  retval;
       int *rfd;
       int *wfd;
       int  err;
     };
     
     std::list<pipe_data> pipe_actions;
     
     struct fork_data
     {
       fork_data(pid_t retval_, int err_)
         : retval(retval_), err(err_) {}
       pid_t retval;
       int   err;
     };
     
     std::list<fork_data> fork_actions;
     
     struct read_data
     {
       read_data(ssize_t retval_, int *efd, const void *data_, int err_)
         : retval(retval_), expected_fd(efd), data(data_), err(err_) {}
       ssize_t     retval;
       int        *expected_fd;
       const void *data;
       int         err;
     };
     
     std::list<read_data> read_actions;
     
     struct write_data
     {
       write_data(ssize_t retval_, int *efd, const void *data, size_t len, int err_)
         : retval(retval_), expected_fd(efd), expected_data(data),
           expected_len(len), err(err_) {}
       ssize_t     retval;
       int        *expected_fd;
       const void *expected_data;
       size_t      expected_len;
       int         err;
     };
     
     std::list<write_data> write_actions;
     
     
     struct wait_data
     {
       wait_data(int retval_, pid_t pid_, int err_)
         : retval(retval_), pid(pid_), err(err_) {}
       int        retval;
       pid_t      pid;
       int        err;
     };
     
     std::list<wait_data> wait_actions;
     
     typedef enum { unknown, is_pipe, is_closed } fdstatus;
     std::map<int, fdstatus> file_descriptors;
     
     
     //
     // wrapper implementations
     //
     
     extern "C" pid_t fork() throw ()
     {
       if (fork_actions.empty()) return original::fork();
       const fork_data &a = fork_actions.front();
       errno = a.err;
       int rv = a.retval;
       fork_actions.pop_front();
       return rv;
     }
     
     extern "C" ssize_t read(int fd, void *p, size_t n)
     {
       if (read_actions.empty()) return original::read(fd, p, n);
       const read_data &a = read_actions.front();
       if (a.expected_fd)
         {
           ASSERT_EQ(fd, *a.expected_fd);
         }
       ssize_t rv = a.retval;
       if (rv != -1)
         {
           size_t srv = size_t(rv);
           ASSERT_GE(n, srv);
           memcpy(p, a.data, srv);
         }
       errno = a.err;
       read_actions.pop_front();
       return rv;
     }
     
     extern "C" ssize_t write(int fd, const void *p, size_t n)
     {
       if (write_actions.empty()) return original::write(fd, p, n);
       const write_data &a = write_actions.front();
       if (a.expected_fd)
         {
           ASSERT_EQ(fd, *a.expected_fd);
         }
       ssize_t rv = a.retval;
       if (rv >= 0)
         {
           ASSERT_EQ(n, size_t(rv));
           if (::memcmp(a.expected_data, p, n) != 0)
             {
               FAIL << "write() data differs from expected";
             }
         }
       errno = a.err;
       write_actions.pop_front();
       return rv;
     }
     
     extern "C" int pipe(int fd[2]) throw ()
     {
       int rv;
       if (pipe_actions.empty() || pipe_actions.front().retval == 0)
         {
           rv = original::pipe(fd);
           if (rv == 0)
             {
               file_descriptors[fd[0]] = is_pipe;
               file_descriptors[fd[1]] = is_pipe;
               if (!pipe_actions.empty())
                 {
                   *pipe_actions.front().rfd = fd[0];
                   *pipe_actions.front().wfd = fd[1];
                 }
             }
         }
       else
         {
           rv = pipe_actions.front().retval;
           errno = pipe_actions.front().err;
         }
       if (!pipe_actions.empty()) pipe_actions.pop_front();
       return rv;
     }
     
     extern "C" int close(int fd)
     {
       int rv = original::close(fd);
       ASSERT_NE(file_descriptors[fd], is_closed);
       file_descriptors[fd] = is_closed;
       return rv;
     }
     
     extern "C" pid_t wait(void *p)
     {
       if (wait_actions.empty()) return original::wait(p);
       const wait_data &a = wait_actions.front();
       int rv = a.retval;
       errno = a.err;
       *static_cast<int*>(p) = a.pid;
       wait_actions.pop_front();
       return rv;
     }
     
     
     TEST(fork_fails, EXPECT_EXCEPTION(work::fork_exception))
     {
       fork_actions.push_back(fork_data(-1, ENOMEM));
       work obj;
     }
     
     TEST(pipe_fails, EXPECT_EXCEPTION(work::pipe_exception))
     {
       pipe_actions.push_back(pipe_data(-1, 0, 0, EMFILE));
       work obj;
     }
     
     static const char parent_str[] = "hello parent!";
     
     struct read_fixture
     {
       read_fixture()
         :  len(sizeof(parent_str) - 1),
            zero(0)
       {
         pipe_actions.push_back(pipe_data(0, &rfd, &wfd, 0));
         write_actions.push_back(write_data(sizeof(len),  &wfd, &len,
                                            sizeof(len), 0));
         fork_actions.push_back(fork_data(0x3ffff3, 0));
         wait_actions.push_back(wait_data(0, 0x3ffff3, 0));
     
         read_actions.push_back(read_data(sizeof(len),  &rfd, &len, 0));
         read_actions.push_back(read_data(len,          &rfd,  parent_str,  0));
         read_actions.push_back(read_data(sizeof(zero), &rfd, &zero,        0));
       }
       const ssize_t len;
       const ssize_t zero;
       int rfd;
       int wfd;
     };
     
     TEST(read_one_string, read_fixture)
     {
       {
         work obj;
         ASSERT_EQ(file_descriptors[wfd], is_closed);
         std::string s = obj.get_data();
         ASSERT_EQ(s, parent_str);
         s = obj.get_data();
         ASSERT_EQ(s, "");
         obj.wait();
       }
       ASSERT_EQ(file_descriptors[rfd], is_closed);
     }
     
     TEST(signal_on_string_read, read_fixture)
     {
       read_actions.insert(++read_actions.begin(),
                           read_data(-1, &rfd, 0, EINTR));
     
       {
         work obj;
         ASSERT_EQ(file_descriptors[wfd], is_closed);
         std::string s = obj.get_data();
         ASSERT_EQ(s, parent_str);
         s = obj.get_data();
         ASSERT_EQ(s, "");
         obj.wait();
       }
       ASSERT_EQ(file_descriptors[rfd], is_closed);
     }
     
     TEST(normal_work)
     {
       work obj;
       std::string s = obj.get_data();
       INFO << "s=\"" << s << "\"";
       ASSERT_EQ(s, "");
       obj.wait();
     }
     
     int main(int argc, char *argv[])
     {
       return crpcut::run(argc, argv);
     }

The result of running the test program is:


     FAILED!: signal_on_string_read
     phase="running"  --------------------------------------------------------------
     samples/process-test.cpp:272
     ASSERT_EQ(s, parent_str)
       where s = _____________
             parent_str = hello parent!
     -------------------------------------------------------------------------------
     ===============================================================================
     FAILED!: normal_work
     phase="child"  ----------------------------------------------------------------
     samples/process-test.cpp:284
     s=""
     -------------------------------------------------------------------------------
     ===============================================================================
     5 test cases selected
     
                    Sum   Critical   Non-critical
     PASSED   :       3          3              0
     FAILED   :       2          2              0

The stubbed tests worked out well, except for signal_on_read, but that was pretty obvious from reading the code. But why did the test with the real library functions fail?

There are so many things wrong with this case, it's actually a race for several failures:

Fortunately crpcut is robust against this kind of abuse, so no harm is done. Unfortunately the details of the report aren't too impressive, but they at least hint about the cause of the error.



[1] You can't easily provide your own dlopen(), dlsym() or dlclose() however.

[2] crpcut provides its own malloc(), calloc(), realloc(), free(), operator new(), operator delete(), operator new[]() and operator delete[]() for instrumentation purposes. See the chapter on Heap Management.

You may use or test your own heap implementation, but then you must link with -lcrpcut_basic and lose crpcut's instrumentation support.