Clean tests produce clean code. It’s close to impossible to write clean tests for dirty code.
Do you agree with above? If you do (which I sincerely hope so) read on about a brand new concept in Mockito, the “strict stubbing” feature. In a nutshell it provides:
- better productivity by detecting incorrect stubbing early (covered in great detail in this article).
- cleaner tests by eliminating unnecessary stubbings (somewhat covered, assuming it is obvious)
- DRY (Don’t Repeat Yourself) test code by verifying stubbing automatically (sadly, not covered in this article, if you want to know about it more leave me a comment :).
Stubbing in Mockito is considered delightfully simple and intuitive:
//given
List mock = mock(List.class);
given(mock.get(3)).willReturn("ok");
//expect
assert mock.get(1) == null;
assert mock.get(3) == "ok";
"Things should be as simple as possible, but not simpler" said Albert Einstein. Mockito stubbing is very close to Enstein’s simplicity boundary. There are times when returning null or empty values when stubbing argument does not match causes a lot of pain to our users. The user can stare at the code not understanding why mocks don’t behave as they are configured to. It happens when mocks return nulls instead of stubbed values configured in the test. The user resorts to debugging and stepping program execution to figure out what’s going on.
I will stop here because I feel uncomfortable complaining about the API I designed myself… I hope that "stubbing argument mismatch” scenario does not impact your productivity too often and you still enjoy cutting some great tests with Mockito.
When you hit stubbing argument mismatch scenario with Mockito 1.x here are your resolution options:
- reviewing code: double checking the arguments passed to mocked method in the test and in the code
- debugging: stepping program execution, inspecting the value of arguments. Using debug statements
- reducing complexity: temporarily removing logic to oversimplify the code, temporarily replacing stubbing arguments with permissive argument matchers like "any()"
- leveraging Mockito API: Mockito.RETURNS_SMART_NULLS
- migrating to latest Mockito! (finally I get my punchline)
Mockito 2.1 comes with
JUnit Rule that prints warnings to the console when stubbing argument mismatch is detected. When you stub a method with some argument(s) but then the same method is invoked with different argument(s), a warning is printed. Mockito’s painstakingly comprehensive Javadoc documents this behavior in
MockitoHint class. (BTW. MockitoHint is a marker interface, existing only for documentation and Javadoc linking. Isn’t it a neat idea?)
The reason Mockito produces a warning instead of an immediate test failure is because... it’s really a “warning" not an “error". Sometimes, it is actually desired to call the stubbed method many times, with different arguments, some of those args match and some don’t. Arguably, those scenarios are quite rare. Originally I even had an example here in this blog post describing this use case in more detail. Eventually, I got rid of the example, it was not compelling enough. Take my word for it: there are legit cases where stubbing argument mismatch is _not_ an error. I failed to produce an example for this use case, perhaps you can share an something from your tests?
Mockito 2.1 also contains improved behavior of
JUnit Runner which detects unused stubbings and reports them as failures. This helps keeping the tests clean. This feature is not directly connected to stubbing argument mismatch scenario. However, it is a part of the same design thought - making Mockito stubbing stricter for cleaner tests and improved productivity.
Just like the JUnit Rule, the Runner also prints warnings on stubbing argument mismatch. Have you noticed those warnings? Do you find them useful? Here are the traits of the warning system:
- Warnings are useful only when developers actually pay attention to the console output.
- Program output can interleave with the warnings. Given enough noise in the output the warnings will simply be lost.
- Warnings can interleave with actual test failure which can lead to confusing test result. The user sees both: the warning and the error and tries to figure out what is the problem to fix.
To address the caveats with warnings, Mockito 2.3 comes with "strict stubbing" option for JUnit Rules. This feature makes the stubbing argument mismatch fail fast. The user immediately gets feedback that there something fishy about the stubbing in test _or_ the use of stubbed method in the production code. This new fail-fast behavior might be undesired for certain use cases therefore strict stubbing can be turned off per test. Strict stubbing is tentatively planned as default behavior for Mockito version 3 because we believe it improves productivity. We would really appreciate if you tried out strict stubbing and let us know how it works for you.
Strict stubbing with JUnit Rules:
@Rule public MockitoRule mockito = MockitoJUnit.rule().strictness(Strictness.STRICT_STUBS);
Mockito 2.5.2 adds strict stubbing support for JUnit Runner:
@RunWith(MockitoJUnitRunner.StrictStubs.class)
public class SomeTest {
//...
}
Currently strict stubbing in Mockito is only available with JUnit Rule or JUnit Runner. This is not a great news for TestNG fans or users that for some unknown reason decided to not use Mockito’s JUnit support (Why not? It’s really good for you. Let us know why!). We currently work on adding strict stubbing support without the need to use JUnit Runner or JUnit Rule. It will unlock the feature to everybody. This feature will be merged within few days so it’s your last chance to provide feedback to the
design note or the
pull request.
If you got that far I can only thank you for reading. Let us know what you think about
strictness in Mockito.
May your tests be blessed with clarity!
