How Mock Can Improve Your Unit Tests: Part 1

We've written in the past about the importance of unit tests and how to write better ones: Culture of Unit Testing, Subtests are the Best.

Mocking is a little tricky to learn to use, but it's really handy to have in your testing toolbox. The purpose of mocking is to focus each unit test by faking execution of all code other than the small piece that the unit test is trying to verify.

Let's start with a small example. Suppose you have a utility function that calls an API with some parameters, does a little manipulation of the return value, and returns part of it. You don't want to call the API every time you run the test of this function — it might be expensive to run, or slow, or difficult to set up.

Instead, you mock the call to the API, and make it look to the function being tested as if the API had returned whatever value you want. You can verify both that the function returns the value it should, and that the function passed the expected parameters to the API call, without ever calling the API.

You can also make the API appear to fail, even raise an exception, something you could not test by actually calling the API.

This two-part series of blog posts will teach you how to use Python's unittest.mock module to add mocking to your unit tests.

The Mock Methods and Mock Objects

There are usually two different aspects of mocking going on at the same time, and it's helpful to understand the difference.

First, we can temporarily change the value of something (called “patching”) using mock.patch or mock.patch.object, then restore its original value.

Second, we can use mock objects, which are Python objects that can be used to simulate another object, and control the behavior of the simulated object.

Often when we use mocking, we're using the patch methods to temporarily replace something with a mock object. But sometimes we can use patching or a mock object independently.

Mock a Function Being Called in Another Module

It's common to test a function that is calling another function that it has imported. Example:

## package1/code.py
from package2.subpackage import foobar

def function_to_test(a, b):
    return foobar(a, b + 2) + "xyz"

Here's the pattern we use to test function_to_test, while mocking the call to foobar:

from django.test import TestCase
from unittest import mock
from package1.code import function_to_test

class TestClass(TestCase):
    def test_normal_call(self):
        with mock.patch("package1.code.foobar") as mock_foobar:
            mock_foobar.return_value = "something"
            retval = function_to_test(1, 2)

        self.assertEqual("somethingxyz", retval)
        mock_foobar.assert_called_with(1, 4)

Let's break this down.

On line 9, we call the function with some arguments and save the return value. Then on line 11, we check that we got back the return value we expected.

On line 7, mock.patch("package1.code.foobar") indicates that we want to mock the symbol foobar in the module package1.code. In other words, while this mock is in effect, we're going to change the value of foobar within package1.code, and afterward we will restore its original value.

Notice that while foobar is in package2.subpackage, we do the mock on package1.code.foobar, which is the reference that is being called by our function under test! It wouldn't do any good to change things in package2.subpackage anyway, since package1.code got a reference to the original foobar when it was imported, and that's what will be called unless we change that reference.

By default, mock.patch() will create a new mock object and use that as the replacement value. You can pass a different object using mock.patch(new=other_object) if you want it to be used in place of a newly created mock object.

By using the mock as a context manager, we limit its scope to the short time we need it to be in effect. as mock_foobar saves the mock object itself to the mock_foobar variable so we can manipulate it. We're going to want to do two things with that mock object.

First, we control the return value when the mock object is called by assigning to its return_value attribute. While this mock is in effect, any call to foobar from within package1.code will immediately return the value "something".

Second, we're going to verify that foobar was called with the arguments we expect — in this case, (1, 4). mock_object.assert_called_with(*args, **kwargs) asserts that the last call to the mock object was passed (*args, **kwargs).

(Before I started using mocking so much, I always assumed that when using a context manager — with <something> as <varname> — that <varname> went out of scope outside the with block. That turns out not to be the case, and it's very handy, as we can limit the time our mocking is applied (within the with block) but still reference the mock object afterward to see what happened to it.)

Where to Mock

It can be confusing figuring out what to pass to mock.patch to get the mocking behavior we need.

In the most common case, we have a module that imports some object and then calls or otherwise accesses it, and we want the code in the module to see a mocked version of that object instead of the real one. This is the module containing the code that we are testing.

In that case, we want to identify the module with its full package name, e.g. "ourpackage.ourmodule", and combine it with the name of the object to be mocked as it appears in that module.

So if the module has from urllib.urlparse import urlparse as parse_method, then we need to pass ourpackage.ourmodule.parse_method to mock.patch.

If instead of importing an object, we define a function or variable in the same module, we can mock it the same way as if it was imported into our module.

When we Can't Mock

There are some common cases where we can't use mocking. It's okay to rewrite the code being tested a little bit in order to make it possible to use mocking when testing it. I'll usually add a comment to explain why the code is maybe a little less straightforward than a reader might expect.

In the first case, something is being imported from a C language module and we can't mock that. For example, if we have from datetime import timedelta, we can't mock timedelta in that module because datetime is a C module, not a Python module.

If we need to, we can wrap that in a Python function and mock our function. E.g.:

from datetime import timedelta

def daydelta(days):
    return timedelta(days=days)

then we can mock daydelta. Of course, then we have to be sure to use daydelta in our code instead of calling timedelta directly, or else we might not really be using what we're testing.

In the second case, the thing we want to mock isn't at the top level of the module, maybe because we're importing it inside a function or class. mock.patch can only mock objects at the top level of modules.

Again, we might write a little Python function that uses the thing we'd otherwise mock, and mock the Python function instead.

I recently ran into another case. A mocked object method was supposed to be called from a Django template, but the Django template code didn't recognize the mock object as callable and tried to just use the mock object directly. I ended up finding a different place to mock, that wasn't being called directly from the template.

Part 2

In Part 2 of this series, I'll show more cases where you can apply mocking, present a model of how mocking works, and talk more about features of Python's mock objects. Read Part 2 now.

For More Information

I've never found the mock documentation particularly clear, but once you know some of the basics from this post, I hope you'll be able to approach them more usefully.