Ever had a child that asked, “Why?” to every answer you give. It could have started with your statement, “Children should do what their parents tell them to do.” If you try to answer each “Why?” thoroughly, you must psychoanalyze each answer until eventually you’re describing the meaning of the Universe. If that’s what is seems like when learning python’s “from unittest.mock patch, MagicMock”, you’ve come to the right place.
In this article I’m going to unravel the mystery of patch and MagicMock. If at first it seems you need the meaning of the Universe to get it, don’t worry, it’s not that hard. Once I got over this hurdle, writing unit_tests has become second nature. Read on and you can finally know “Why”.
Background
patch is used to override a method and make it return whatever you want. This is very useful in testing, so you can create situations that are difficult to do naturally, or for preventing external requests. The latter is the case I’m going to present here. It’s good practice to prevent external calls, so your unit tests don’t change things. You should be able to run
python manage.py test
while offline, and if you’re online, nothing external should get called.
To setup the example, say you have a Django model and you want to add a private method, _get_page, that returns a requests.request response object. You could use this method to test the value of the “name” field exists on the page residing on any URL, for example.
You’re going to create a new Django project, add an example app, write unit tests with the magic use of MagicMock, then write the code that satisfies the unit tests. I’ll explain the “Why” at the end. Follow along and learn how to use patch and MagicMock in combo.
You can find the code for this example at https://github.com/djangopractice/doublemagic
Setup
mkvirtualenv -p /usr/local/bin/python3.4 doublemagic workon doublemagic pip install django pip install requests django-admin.py startproject doublemagic cd doublemagic django-admin.py startapp example
Connect example to INSTALLED_APPS
In the doublemagic/settings.py file, add ‘example’ to INSTALLED_APPS:
...
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'example',
)
....
Run python manage test
Haven’t written any tests or code yet, just make sure everything is setup right before we start.
$ python manage.py test Creating test database for alias 'default'... ---------------------------------------------------------------------- Ran 0 tests in 0.000s OK Destroying test database for alias 'default'...
Write unit tests
Test Driven Development is a great practice. Let’s write our tests first, see them fail, then write the code to satisfy the tests.
Edit the example/tests.py file:
from django.test import TestCase
from unittest.mock import patch, MagicMock
from example.models import Example
class ExampleTestCase(TestCase):
def setUp(self):
self.example = Example.objects.create(name='Example 1', description='The first example')
def test_example_exists(self):
self.assertEqual(self.example.description, 'The first example')
@patch('example.models.request')
def test_get_page(self, req):
url = MagicMock()
self.example._get_page(url)
req.assert_called_once_with('GET', url)
@patch('example.models.Example._get_page')
def test_name_in_page_calls_get_page(self, getpage):
url = MagicMock()
self.example.name_in_page(url)
getpage.assert_called_once_with(url)
@patch('example.models.Example._get_page')
def test_name_in_page(self, getpage):
getpage.return_value = MagicMock(
content='Text that contains {} in it'.format(self.example.name))
self.assertTrue(self.example.name_in_page(MagicMock()))
@patch('example.models.Example._get_page')
def test_name_in_page_not(self, getpage):
getpage.return_value = MagicMock(content='Text that does not contain the name')
self.assertFalse(self.example.name_in_page(MagicMock()))
@patch('example.models.Example._get_page')
def test_name_in_page_closes_response(self, getpage):
resp = MagicMock()
getpage.return_value = resp
self.example.name_in_page(MagicMock())
resp.close.assert_called_once_with()
Don’t worry, I’ll explain what’s going on in a bit. For now, just type it in and try it out.
Run python manage test, FAIL
This time, unit tests are going to fail. We haven’t written the code yet, only the unit test. We expect this to fail:
$ python manage.py test
Creating test database for alias 'default'...
E
======================================================================
ERROR: example.tests (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
...
from example.models import Example
ImportError: cannot import name 'Example'
----------------------------------------------------------------------
Ran 1 test in 0.002s
FAILED (errors=1)
Destroying test database for alias 'default'...
Write the Code
Now that we have failing unit tests, we must write the code to satisfy the tests.
Edit the example/models.py file:
from django.db import models
from requests import request
class Example(models.Model):
name = models.CharField(max_length=100)
description = models.TextField()
def _get_page(self, url):
return request('GET', url)
def name_in_page(self, url):
resp = self._get_page(url)
resp.close()
return self.name in str(resp.content)
Run python manage test, PASS
This time, the tests are satisfied:
$ python manage.py test Creating test database for alias 'default'... ...... ---------------------------------------------------------------------- Ran 6 tests in 0.007s OK Destroying test database for alias 'default'...
Why?
The “test_example_exists” test is just forcing the Example model to exist with a name and description.
Let’s look close at the next 5 tests, the last does the double magic that I really want to explain. The first four also have good lessons in them.
test_get_page
@patch('example.models.request')
def test_get_page(self, req):
url = MagicMock()
self.example._get_page(url)
req.assert_called_once_with('GET', url)
This makes sure that Example._get_page(self, url) exists and that it calls the requests.request method. The “from requests import request” in the example/models.py file makes it so you can patch ‘example.models.request’ – this is the exact request you want to patch. It’s represented by “req” in this test. req.assert_called_once_with(‘GET’, url) is saying that example.models.request, which is requests.request, is called with (‘GET’, url). Because it’s patched, requests.request is NOT going to actually get called, but rather our req object. Because req is a MagicMock, you can pass it whatever you want, like another MagicMock. In this test, we pass it “url”, which we’ve made a MagicMock(), so we can test that get_page passes it to request. Confused yet? Keep asking why! This isn’t even the part I wanted to show you. This is the prerequisite to the hard part.
test_name_in_page_calls_get_page
@patch('example.models.Example._get_page')
def test_name_in_page_calls_get_page(self, getpage):
url = MagicMock()
self.example.name_in_page(url)
getpage.assert_called_once_with(url)
This test makes sure the “name_in_page” method uses the Example._get_page method. It’s using the same strategy as the previous test. This doesn’t make sure it does anything with it, just that it’s called. The next two tests will make sure it properly tests the response.
test_name_in_page
@patch('example.models.Example._get_page')
def test_name_in_page(self, getpage):
getpage.return_value = MagicMock(
content='Text that contains {} in it'.format(self.example.name))
self.assertTrue(self.example.name_in_page(MagicMock()))
When we call name_in_page, it’ll attempt to call _get_page, which is patched as getpage. Our getpage will return a MagicMock that has a content attribute defined as a string with the example name in it. We can now assert that it returns true with getpage, which represents Example._get_page, returns a string with the example name in it.
test_name_in_page_not
@patch('example.models.Example._get_page')
def test_name_in_page_not(self, getpage):
getpage.return_value = MagicMock(content='Text that does not contain the name')
self.assertFalse(self.example.name_in_page(MagicMock()))
This is the negative of the previous case. This is important, because without it, name_in_page could blindly return True. Since name_in_page returns a Bool, need to test both the True and False cases.
test_name_in_page_closes_response
@patch('example.models.Example._get_page')
def test_name_in_page_closes_response(self, getpage):
resp = MagicMock()
getpage.return_value = resp
self.example.name_in_page(MagicMock())
resp.close.assert_called_once_with()
This is the double magic. It’s double because we’re making the return_value be another Magic Mock, that is a MagicMock object, resp. Even the previous two were double magic, but the difference with this one is saving an instance of the MagicMock as resp. By doing so, we can assert that a method is called on this object, inside the name_in_page method.
This test makes sure that “close()” is called on the returned response object.
It really isn’t necessary to call resp.close(), but this was the most simplified version of the example I could think of. You’ll run into cases where this tactic comes in handy. More than handy, required.
Test With Shell
Let’s run the shell and see the example in action. This is not unit testing, this is manual verification. You could take this and turn into automated integration tests.
It’s really important to do this. It’s better to have automated integration tests that make sure all the dependencies of your application work together, but that’s beyond the scope of this article. For now, let’s just try it out and see it work:
$ python manage.py shell
Python 3.4.2 (default, Oct 16 2014, 05:21:12)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.51)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from example.models import Example
>>> example = Example.objects.create(name='W3')
>>> example.name_in_page('http://koopman.me/')
True
>>> example = Example.objects.create(name='QWERTYUIOOOOP')
>>> example.name_in_page('http://koopman.me/')
False
>>>
Conclusion
I hope this exercise has helped you. Understanding how to use patch and MagicMock is critical to Test Driven Development. Remember, you should be able to run unit tests offline, and even if you’re online, nothing external should be called. Use patch to prevent external calls from occurring, and to ensure methods are being called.
Once you understand this concept, consider triple magics. Sometimes you need your patch to return a MagicMock() with an attribute that returns another MagicMock() that has an attribute or method to test. Real brain teaser. Code on.