Library to fake intermittent failures according to tester-defined policy?
- by crosstalk
I'm looking for a library that I can use to help mock a program component that works only intermittently - usually, it works fine, but sometimes it fails.
For example, suppose I need to read data from a file, and my program has to avoid crashing or hanging when a read fails due to a disk head crash. I'd like to model that by having a mock data reader function that returns mock data 90% of the time, but hangs or returns garbage otherwise. Or, if I'm stress-testing my full program, I could turn on debugging code in my real data reader module to make it return real data 90% of the time and hang otherwise.
Now, obviously, in this particular example I could just code up my mock manually to test against a random() routine. However, I was looking for a system that allows implementing any failure policy I want, including:
Fail randomly 10% of the time
Succeed 10 times, fail 4 times, repeat
Fail semi-randomly, such that one failure tends to be followed by a burst of more failures
Any policy the tester wants to define
Furthermore, I'd like to be able to change the failure policy at runtime, using either code internal to the program under test, or external knobs or switches (though the latter can be implemented with the former).
In pig-Java, I'd envision a FailureFaker interface like so:
interface FailureFaker {
/**
Return true if and only if the mocked operation succeeded.
Implementors should override this method with versions consistent
with their failure policy.
*/
public boolean attempt();
}
And each failure policy would be a class implementing FailureFaker; for example there would be a PatternFailureFaker that would succeed N times, then fail M times, then repeat, and a AlwaysFailFailureFaker that I'd use temporarily when I need to simulate, say, someone removing the external hard drive my data was on. The policy could then be used (and changed) in my mock object code like so:
class MyMockComponent {
FailureFaker faker;
public void doSomething() {
if (faker.attempt()) {
// ...
} else {
throw new RuntimeException();
}
}
void setFailurePolicy (FailureFaker policy) {
this.faker = policy;
}
}
Now, this seems like something that would be part of a mocking library, so I wouldn't be surprised if it's been done before. (In fact, I got the idea from Steve Maguire's Writing Solid Code, where he discusses this exact idea on pages 228-231, saying that such facilities were common in Microsoft code of that early-90's era.) However, I'm only familiar with EasyMock and jMockit for Java, and neither AFAIK have this function, or something similar with different syntax.
Hence, the question: Do such libraries as I've described above exist? If they do, where have you found them useful? If you haven't found them useful, why not?
