Fakes are Better than Mocks

Building any sizable software project involves writing components which take dependencies on other components. In order to write unit tests for a component with dependencies, the unit test must be able to inject those dependencies. This article will focus on two types of dependencies that can be used in unit tests — fakes and mocks. It will make the claim that using fakes should be preferred over the using mocks in almost all cases.

A mock does not provide a working implementation of a dependency. It simply defines the expected calls, inputs and outputs of the dependencies. Mocks cannot be used to actually run the system end to end, but they can simulate behavior in order to test isolated units.

Fakes on the other hand are fully working implementations. A fake could be used to run the whole system end to end. Fakes will typically take some implementation shortcut which makes them simpler to implement but also makes the unsuitable for use in a real production environment.

In this article we are going to expand the definition of fakes to also include using the real dependencies in unit tests. There are cases in unit tests where the real implementation of a dependency can be wired up and used directly. So even though it’s a bit non-traditional, this article will lump any type of fully working implementation (real or not) into the category of fake dependencies. This terminology shortcut will make the rest of the article easier to follow without watering down the central claim of the article.

When developers write unit tests they will likely end up naturally using some combination of fakes and mocks throughout their codebase. Some developers will lean towards one extreme of using mocks for every dependency, while others will lean towards the other extreme of trying to use fakes for everything. Without deliberate consideration of the choices that are being made, developers are likely to pick the easiest option which is often extensive use of mocks.

Until recently, I did not think deliberately about the type of test dependencies I was picking in unit tests. I just picked whatever was easiest and ran with it. But a recent conversation with a coworker encouraged me to reconsider. I now have a strong preference towards using fakes, over mocks in nearly all cases.

Using fakes nearly always requires more initial work to setup than using mocks, but I believe it provides significant benefits that justify the added upfront setup cost. I am not religious on this topic, I will not claim that mocks should never be used, but I have recalibrated my default to reach for fakes as my first option.

Now lets dive into why I had this recalibration

Generally there are two types of test assertions that are made in unit tests — behavioral assertions and state assertions. Behavioral assertions, assert that the system did an operation in a certain way. State assertions, assert that the system is in a given state.

Unit tests that use mocks, will primarily make behavioral assertions not state assertions. Mocks dictate how each dependency should be used by the system under test (SUT). They do this by specifying several attributes of how the dependencies will be invoked — the number of calls that will be made, the order in which calls will occur, the inputs / outputs of those calls. By the time a function is tested completely with mocks, the test will essentially look like a blueprint for how the function should preform its job.

Unit tests that use fakes, will primarily make state assertions not behavioral assertions. Fakes do not care about how each dependency gets used at all. Fakes are injected into a unit test so that the SUT can preform its job, not to dictate how the SUT should do its job. Therefore the type of assertions that end up getting made are on the state of the SUT and its dependencies to ensure the job was done correctly.

State assertion based unit tests will look much more like clients of the SUT than will behavioral assertion based unit tests. Clients of the SUT will wire up some dependencies, create the SUT and operate on the SUT by calling read/write methods. State assertion based unit tests will do exactly the same thing, the only difference is how the dependencies get created. In contrast, behavioral assertion based unit tests will not look anything like client code. Behavioral assertion based unit tests will describe how the SUT should preform every action. They do this by specifying the mock behaviors.

Using fakes in unit tests rather than mocks will result in the tests making state based assertions instead of behavioral based assertions. This in turn will make the unit tests look more like client code. These outcomes have several advantages —

• Documentation: The unit test code can act like documentation to clients who wish to integrate with the the SUT. The unit tests will read so much like client code, that looking at them will make it clear what the client facing contract is.
• Refactoring Confidence: When behavioral based assertions are the central way in which code is tested, the tests will almost certainly need to be updated during a refactor. If the order of any calls is changed or the abstraction boundary of any dependencies is changed the unit tests will break, and need to be updated along side the refactor. State based assertions on the other hand, should not break during a refactor because similar to clients it does not matter how the SUT does its job as long as the state achieved at the end of the execution satisfies the contract. Having unit tests which are resilient to refactors, increases confidence that the refactor was done correctly and without client visible changes.
• Readability: Tests which require setting up mocks can become hard to read as complexity increases. When testing complex components, the wiring up the expected mock calls can end up being the vast majority of the total lines of testing code. In contrast there is typically little to no setup when using fakes, just like client code the dependencies are wired up and then you are off to the races.
• Better Abstraction Design: When unit tests resemble client code, it encourages writing more clear abstractions. State based assertions essentially force the test author to dog food the abstraction of the SUT. This is great because it makes abstraction inconsistencies more obvious and more likely to be fixed.
• Integration Tests Reusability: When fakes are used instead of mocks throughout unit tests it ends up actually saving effort in the long run, because eventually fakes will need to be used anyway in order to write end to end integration tests. If unit tests are already setup using fakes, then those same fakes can simply be reused for integration tests. This is not true if unit tests are using mocks to make behavioral based assertions.

Given these benefits, I will always try to reach for fakes before mocks as my first option.

I have not seen or thought of many cases in which I would have to give up my love of fakes and use mocks instead, but one semi-compelling example is when the the interesting attribute of the test dependency is its behavior rather than its state. The best example I have come up with for this is a cache. A cache explicitly is hard to test with state based assertions because it returns the same “state” regardless of if there was a cache hit or a cache miss. But unit test authors likely want to know if the cache is being hit. Here it is the behavior that is explicitly interesting not the state.

Even this example is not super compelling though because its possible to expose private state on the results return from the cache which indicate if they resulted from a cache hit or a cache miss, so even in this example its likely possible to use fakes instead of mocks.

I will keep thinking about cases in which fakes cannot be used, but for now that is all I got on this topic. Signing off until next time.