Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[PyCon US 2016] To mock or not to mock, that is...

[PyCon US 2016] To mock or not to mock, that is the question

Mocking is a very powerful testing concept that has some dangerous pitfalls. There are obvious use cases where mocks are an absolute requirement to be able to test a part of the app. Nevertheless sometimes apparently useful mocks can yield erroneous test results. This talk goes into deeper detail on the trade-off of using mocks in testing.

Ana Balica

May 30, 2016
Tweet

More Decks by Ana Balica

Other Decks in Programming

Transcript

  1. ✓ Setup ✓ Test ✓ Verify state ✓ Teardown ✓

    Setup ✓ Setup expectations ✓ Test ✓ Verify expectations ✓ Verify state ✓ Teardown Stubs Mocks
  2. Mock >>> from mock import Mock >>> m = Mock()

    >>> m.foo = 1 >>> m.foo 1 >>> m.bar <Mock name='mock.bar' id='4310136016'>
  3. @patch.object(DataLoader, '_get_file_contents') def test_parse_json_from_file(self, mock_def): mock_def.return_value = ('{"a": 1, "b":

    2, "c": 3}', True) output = self._loader.load_from_file('dummy_json.txt') self.assertEqual(output, dict(a=1, b=2, c=3)) json.loads IO operations
  4. @mock.patch('time.sleep') def test_500_retry(self, sleep_mock): self.set_http_response(status_code=500) # Create a bucket, a

    key and a file with self.assertRaises(BotoServerError): k.send_file(fail_file) Clocks, time, timezones time.sleep
  5. ✓ System calls ✓ Streams ✓ Networking ✓ IO operations

    ✓ Clocks, time, timezones ✓ Unpredictable results
  6. with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data) self.assertTrue(form.is_valid())

    saved_pony = form.save() self.assertTrue(saved_pony.age, 3) mock_save.assert_called_once() Problems?
  7. with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data) self.assertTrue(form.is_valid())

    saved_pony = form.save() self.assertTrue(saved_pony.age, 3) mock_save.assert_called_once() Problems?
  8. with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data) self.assertTrue(form.is_valid())

    saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.assert_called_once() Problems?
  9. with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data) self.assertTrue(form.is_valid())

    saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.assert_called_once() Problems?
  10. with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data) self.assertTrue(form.is_valid())

    saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.assert_called_twice() Problems?
  11. with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data) self.assertTrue(form.is_valid())

    saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.make_me_sandwich() Problems?
  12. Solution 1 with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony,

    data=data) self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.assert_called_once_with()
  13. Solution 2 with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony,

    data=data) self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) self.assertEqual(mock_save.call_count, 1)
  14. Failure with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data)

    self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) self.assertEqual(mock_save.sandwich_count, 1)
  15. Problems? with patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data)

    self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) self.assertEqual(mock_save.call_count, 1)