Library to fake intermittent failures according to tester-defined policy?

Posted by crosstalk on Programmers See other posts from Programmers or by crosstalk
Published on 2011-03-03T05:52:32Z Indexed on 2011/03/03 7:32 UTC
Read the original article Hit count: 317

I'm looking for a library that I can use to help mock a program component that works only intermittently - usually, it works fine, but sometimes it fails.

For example, suppose I need to read data from a file, and my program has to avoid crashing or hanging when a read fails due to a disk head crash. I'd like to model that by having a mock data reader function that returns mock data 90% of the time, but hangs or returns garbage otherwise. Or, if I'm stress-testing my full program, I could turn on debugging code in my real data reader module to make it return real data 90% of the time and hang otherwise.

Now, obviously, in this particular example I could just code up my mock manually to test against a random() routine. However, I was looking for a system that allows implementing any failure policy I want, including:

  • Fail randomly 10% of the time
  • Succeed 10 times, fail 4 times, repeat
  • Fail semi-randomly, such that one failure tends to be followed by a burst of more failures
  • Any policy the tester wants to define

Furthermore, I'd like to be able to change the failure policy at runtime, using either code internal to the program under test, or external knobs or switches (though the latter can be implemented with the former).

In pig-Java, I'd envision a FailureFaker interface like so:

interface FailureFaker {
    /**
        Return true if and only if the mocked operation succeeded.
        Implementors should override this method with versions consistent
        with their failure policy.
     */
    public boolean attempt();
}

And each failure policy would be a class implementing FailureFaker; for example there would be a PatternFailureFaker that would succeed N times, then fail M times, then repeat, and a AlwaysFailFailureFaker that I'd use temporarily when I need to simulate, say, someone removing the external hard drive my data was on. The policy could then be used (and changed) in my mock object code like so:

class MyMockComponent {
    FailureFaker faker;

    public void doSomething() {
        if (faker.attempt()) {
            // ...
        } else {
            throw new RuntimeException();
        }
    }

    void setFailurePolicy (FailureFaker policy) {
        this.faker = policy;
    }
}

Now, this seems like something that would be part of a mocking library, so I wouldn't be surprised if it's been done before. (In fact, I got the idea from Steve Maguire's Writing Solid Code, where he discusses this exact idea on pages 228-231, saying that such facilities were common in Microsoft code of that early-90's era.) However, I'm only familiar with EasyMock and jMockit for Java, and neither AFAIK have this function, or something similar with different syntax.

Hence, the question: Do such libraries as I've described above exist? If they do, where have you found them useful? If you haven't found them useful, why not?

© Programmers or respective owner

Related posts about testing

Related posts about language-agnostic