TL;DR;

When writing unit tests, test the state of your program, not the implementation. That means, most of the time, avoiding mocks at all costs. Sometimes it can be hard to achieve that but there are a few strategies we can use to avoid the use of mocks.

Unit Tests Refresher

Much has been written on this topic already, but since I continue to see developers making this mistake, I felt like a refresher on why this is a bad practice could be helpful to many.

Let’s start with why this is a problem in the first place.

The goal of unit tests is to make sure that a given component of a program (a unit) behaves as expected. In order to test this component, and only this component, we must isolate it from the rest of our program so we can be sure its behavior (and state) is not influenced by other components it integrates with.

In order to perform our tests, we must treat a component as an opaque box and only interact with it via its interface. It’s pretty much like test-driving a car, where we only need to use the pedals and steering wheel and see if it moves accordingly, but we don’t need to check the engine or other mechanic parts. For the purposes of our test, if the car moves (and brakes), it works.

Going back to computer programs, imagine we want to test a function that sums an array of integers and returns the sum as the result:

public Integer sum(numbers: Array[Integer]) {
//implementation goes here
}

My question to you is, can we write a unit test for this function without knowing how it’s implemented? The answer is, of course we can:

public void shouldReturn10() {
  //given
  var input = new Integer[] {1, 2, 3, 4};

  //when
  var result = sum(input);

  //then
  assertEquals(result, 10);
}

There are many ways we could implement the function sum but from the unit tests perspective it doesn’t matter how we do it as long as we return the correct result. Doing so allows us to change the implementation without impacting our tests.

Imagine for example we decide to delegate the actual computation of the sum to an external web API:

public Integer sum(numbers: Array[Integer]) {
  //web client setup goes here
  return webClient.getRequest(numbers);
}

Ignore for a moment how useless the sum function became. If you want, you can pretend it performs a lot of complicated business logic instead of just returning the result of the api call right away. What I’d like you to focus instead is the fact that it now has an external dependency that will make our test more complicated.

So, how can we test the sum function now? If we use a valid url and the web api is up and running, our test might pass. Otherwise it would fail since it wouldn’t be able to get a response from the api.

We might be tempted to use a valid url and run our test against the “real thing” but then we have a problem:

OUR TEST IS NO LONGER A UNIT TEST

It became an integration test instead. There’s a place and moment for integration tests but since we’re talking about unit tests here, what can we do to keep our test “unitary” without breaking it?

Mocks

If you’re a seasoned developer, you’re probably familiar with the concept of mocks. If not, you can check Martin Fowler’s article for a detailed discussion between mocks and stubs. Before moving forward though, I wanted to clarify what I mean by mock.

To me, mocks are fake implementations of external dependencies that are validated against during tests, with the “validated against” part being important here.

Accordingly to the definition I use, if we have a fake implementation of an external dependency that we don’t validate against, then we’re talking about stubs. Stubs only exist to “fill in the blanks”, or, in other words, to allow our tests to run successfully by learning how to respond to our method calls.

Mocks instead, can have their behavior verified after an interaction with the component under test.

With that knowledge in hand, let’s rewrite our test to make use of mocks:

public void shouldReturn10() {
  //given
  var input = new Integer[] {1, 2, 3, 4};
  
  //mock setup
  when(webClient.getRequest(input)).return(10);

  //when
  var result = sum(input);

  //then
  assertEquals(result, 10);
}

Ok, we have a passing test again, but at what cost?

What if we want to change the request type from a GET to a POST? What if we want to use the asynchronous version of the getRequest method? What if we want to replace the webClient class entirely?

Now our test knows too much and you know what happens to folks who know too much, right? They end up at the bottom of a river or seven feet under the ground.

You might think this is not a big deal because it’s just one test and the change required wouldn’t be that complicated. However, in a real-life project it’s very likely that we would end up with dozens of tests for a single function and hundreds of functions requiring multiple mocks each, where replacing a dependency like our webClient could prove to be a challenge.

Side Effects

Before we discuss solutions to our mocking problem, let’s analyze another common scenario. Imagine now we need to test a function that contains some business logic but doesn’t return a result, that is, it only performs side effects:

public void alert(errorMessage: String) {
  String formattedMessage = "ERROR: " + errorMessage;
  println(formattedMessage);
}

Ok, we can’t. There’s nothing we can (easily) do here. What about this one:

public void alert(errorMessage: String) {
  String formattedMessage = "ERROR: " + errorMessage;
  logger.error(formattedMessage);
}

We could try using a mock. Let’s see how that goes:

public void shouldGenerateAlert() {
  //given
  String inputErrorMessage = "Oh crap!";
  String outputErrorMessage = "ERROR: Oh crap!";

  //mock setup
  when(loggerMock.error(outputErrorMessage)).doNothing();

  //when
  alert(inputErrorMessage);

  //then
  verify(loggerMock.error(outputErrorMessage)).wasCalled();
}

So, that works, I guess. However, what are we testing here?

Well, we’re definitely testing the business logic of the alert function which consists of prepending the string “ERROR: ” to the error message before passing it to the logger component.

We’re also verifying if the the logger component’s error method is being called with the expected argument but then we are back to the previous problem of having a test that knows too much about the things it’s testing.

The issue is that by testing the behavior of our function (the string concatenation bit) we’re forced to test how it’s implemented (the call to the logger component).

What if we did the following:

public class LoggerUtils {
  public static String formatErrorMessage(errorMessage: String) {
    return "ERROR: " + errorMessage;
  }
}

public void alert(errorMessage: String) {
  String formattedMessage = LoggerUtils.formatErrorMessage(errorMessage);
  logger.error(formattedMessage);
}

We’ve moved the business logic of prepending a string to the error message to an external class/method that we can now test without issues. However, we’re still not able to test the fact that the alert function itself must log a formatted message. The only thing we can do here is to test the function’s behavior which means testing its implementation through the use of mocks like we did before.

Our problem stems from the fact there’s no state for us to assert on. Or is there? Let’s ask ourselves, what changes in the world when we generate an alert? Well, somewhere in a server a string gets appended to a file, but we can’t test that (at least not from a unit test).

Spies

One solution to our problem of lack of state is to create an object that can represent that state so we can assert on. For that, we can use a special kind of stub called spy. A spy not just knows how to behave like the dependency we want to stub but also keeps track of the changes in its state:

public interface LoggerInterface {
  public void error(errorMessage: String);
}

public class LoggerSpy implements LoggerInterface {
  private List<String> messages = new ArrayList<String>();

  public List<String> getMessages { return this.messages; }

  public void error(errorMessage: String) {
    messages.add(errorMessage);
  }
}

Here we’re assuming the Logger class implements the LoggerInterface above. It would work the same way if it extended an abstract or non-sealed (non-final) class/method. If none of those are possible, we can always wrap our dependency into a class that we can extend:

public class LoggerWrapper implements LoggerInterface {
  private Logger logger = new Logger();

  public void error(errorMessage: String) {
    logger.error(errorMessage);
  }
}

public void alert(errorMessage: String) {
  String formattedMessage = LoggerUtils.formatErrorMessage(errorMessage);
  logger.error(formattedMessage);
}

Our function under test looks the same but now it’s calling the error method on an instance of the LoggerInterface (via LoggerWrapper) instead of the Logger class.

Another benefit of the wrapper approach is that you isolate the dependency from the code calling it. In our example, if we decide later to switch to a different logger class (or method) we could do so without impacting our tests.

Anyway, no matter how we end up implementing the spy, the consequence is that we can now implement our test that will assert on the state of the Spy:

public void shouldLogErrorMessage() {
  //given
  String inputErrorMessage = "Oh crap!";
  String outputErrorMessage = "ERROR: Oh crap!";

  //spy setup
  var loggerSpy = new LoggerSpy();

  //when
  alert(inputErrorMessage);

  //then
  assertEquals(1, loggeSpy.gettMessages().size());
  assertEquals(outputErrorMessage, loggerSpy.getMessages().get(0));
}

To keep things simple, we can assume in the code above there’s some form of dependency injection going on that makes the loggerSpy available to the logger function.

With this technique, we are able to move away from a behavior-based to a state-based testing approach. Even if we decide to change the component we use for logging, we’ll only need to change the Spy implementation and not the tests.

Complex Stubs

In some cases, creating stubs/spies manually can incur in a lot of work. A few examples are databases, message queues, caches, and the alike.

Thankfully, the software community has come up with fake implementations for most of the technologies we currently use in our projects, even cloud-native ones.

Localstack, for example, is a great solution for running the AWS stack locally. MySQL has an official mock implementation. I bet a search for “[tetchnology] fake mock” will return at least one result for the most widely-adopted technologies.

The good thing about using this kind of fakes is that you don’t need to change one line of code since they behave just like the real thing (mostly). The only thing we must do is to configure our unit tests to point to the local instances of those dependencies.

These fakes come in two flavors. Libraries or packages that we can embed in our tests and more elaborated services we must spin up separately (think containers). The good thing about the embedded libraries is that they don’t require any extra setup when running in a build pipeline as opposed to the container-based approach that usually requires deployment to a cluster. When running tests locally we shouldn’t see many differences between both approaches.

The Real Thing

Finally, depending on the technology (and as a last resort), we can run unit tests against real instances of our dependencies. Similar to the fakes mentioned in the previous section, they don’t require any code changes but might not (most likely not) be suited for tests running in a build pipeline. To be honest, I don’t think those are really an option and I’m already regretting mentioning it, but they might come in handy as a temporary solution while you and your team figure out a better and definitive approach.

Conclusion

In the end, the choice between state-based and behavior-based unit testing is up to you. I really prefer the state-based approach over the behavior-based one due to my bad experience with the latter over the years. I hope the tips in this article help you next time you’re faced with issues related to the use of mocks.

Thanks for reading so far! Feel free to leave a comment or feedback. Until next time!

Note

The code blocks in this post are pseudo-code inspired in Java but are not meant to be correct and/or compile successfully.