Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Integration and Functional Testing with LiveSer...

Julien Phalip
September 05, 2013

Integration and Functional Testing with LiveServerTestCase, Selenium and More.

Talk given at DjangoCon US 2013. Chicago, IL.
The video: http://www.youtube.com/watch?v=BSYg9-sxSjM

Julien Phalip

September 05, 2013
Tweet

More Decks by Julien Phalip

Other Decks in Programming

Transcript

  1. CONNECTED PERSONAL OBJECTS 5/2012 About me... ‣ Worked with Django

    since 2007 (version 0.96). ‣ Django core committer since 2011.
  2. CONNECTED PERSONAL OBJECTS 5/2012 Definitions (subject to debate) ‣ Unit

    testing Ensure that small parts work in isolation.
  3. CONNECTED PERSONAL OBJECTS 5/2012 Definitions (subject to debate) ‣ Unit

    testing Ensure that small parts work in isolation. ‣ Integration testing Ensure that those parts work well together.
  4. CONNECTED PERSONAL OBJECTS 5/2012 Definitions (subject to debate) ‣ Unit

    testing Ensure that small parts work in isolation. ‣ Integration testing Ensure that those parts work well together. ‣ Functional testing Ensure that the application works, from a user’s perspective.
  5. CONNECTED PERSONAL OBJECTS 5/2012 Testing in Django (the traditional way)

    from django.test import TestCase class MyTests(TestCase): def test_hello(self): r = self.client.get('/hello') self.assertTrue('Hello' in r.content)
  6. CONNECTED PERSONAL OBJECTS 5/2012 Pretty good but incomplete... ‣ Does

    not exercise the full http protocol. ‣ Does not allow to test the user interface.
  7. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  8. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  9. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  10. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  11. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  12. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  13. CONNECTED PERSONAL OBJECTS 5/2012 Some important details ‣ Live server

    listens to localhost:8081 by default ‣ Address can be overridden: ./manage.py test --liveserver=localhost:8082 ./manage.py test --liveserver=localhost:9000-9200 ‣ Address accessible from the test’s code: self.live_server_url
  14. CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests ‣ Testing REST

    APIs: import requests class MyTests(LiveServerTestCase): def test_api_user(self): r = requests.get( self.live_server_url + '/api/user/1/') self.assertEqual(r.status_code, 200) self.assertEqual(r.json, {'username': 'john.doe', 'email': '[email protected]'})
  15. CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests ‣ Testing OAUTH

    workflows: import requests from requests_oauthlib import OAuth2 from oauthlib.oauth2 import BackendApplicationClient class MyTests(LiveServerTestCase): def test_oauth(self): backend = BackendApplicationClient('mybackend') oauth2 = OAuth2(client=backend, token=TOKEN) r = requests.get( self.live_server_url + '/oauth', auth=oauth2) self.assertEqual(r.status_code, 200) self.assertTrue('Hello' in r.content)
  16. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded.
  17. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed.
  18. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested.
  19. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested. ‣ Browser compatibility can be tested.
  20. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested. ‣ Browser compatibility can be tested. ‣ It’s fun!
  21. CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ API to become W3C

    standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python.
  22. CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ API to become W3C

    standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python. ‣ Works across multiple browsers (Firefox, Chrome, IE, Safari, Opera)
  23. CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ API to become W3C

    standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python. ‣ Works across multiple browsers (Firefox, Chrome, IE, Safari, Opera) ‣ Easy to install: pip install selenium
  24. CONNECTED PERSONAL OBJECTS 5/2012 from django.test import LiveServerTestCase from selenium.webdriver.firefox.webdriver

    import \ WebDriver class MyTests(LiveServerTestCase): def setUp(self): self.driver = WebDriver() super(MyTests, self).setUp() def tearDown(self): self.driver.quit() super(MyTests, self).tearDown()
  25. CONNECTED PERSONAL OBJECTS 5/2012 from django.test import LiveServerTestCase from selenium.webdriver.firefox.webdriver

    import \ WebDriver class MyTests(LiveServerTestCase): def setUp(self): self.driver = WebDriver() super(MyTests, self).setUp() def tearDown(self): self.driver.quit() super(MyTests, self).tearDown()
  26. CONNECTED PERSONAL OBJECTS 5/2012 from django.test import LiveServerTestCase from selenium.webdriver.firefox.webdriver

    import \ WebDriver class MyTests(LiveServerTestCase): def test_hello(self): self.driver.get( self.live_server_url + '/hello') e = self.driver.find_element_by_id('btn') e.click() ... ...
  27. CONNECTED PERSONAL OBJECTS 5/2012 Testing visuals with Needle ‣ https://github.com/bfirsh/needle

    ‣ Based on Selenium. ‣ Tests visuals by comparing screenshots.
  28. CONNECTED PERSONAL OBJECTS 5/2012 from needle.cases import NeedleTestCase from from

    selenium.webdriver import firefox class MyTests(NeedleTestCase, LiveServerTestCase): driver_command_executor = firefox.ExtensionConnection( "127.0.0.1", firefox.FirefoxProfile()) def test_homepage_menu(self): self.driver.get(self.live_server_url) self.assertScreenshot('ul.menu', 'menu') Testing visuals with Needle
  29. CONNECTED PERSONAL OBJECTS 5/2012 ====================================================================== FAIL: test_homepage_menu (myapp.tests.MyTests) ---------------------------------------------------------------------- Traceback

    (most recent call last): File "myapp/tests.py", line 115, in test_homepage_menu self.assertScreenshot('ul.menu', 'menu') File "/.virtualenvs/myproject/src/needle/needle/cases.py", line 99, in assertScreenshot pass File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/ python2.6/contextlib.py", line 23, in __exit__ self.gen.next() File "/.virtualenvs/myproject/src/needle/needle/cases.py", line 160, in compareScreenshot % (filename, distance)) AssertionError: The saved screenshot for 'menu' did not match the screenshot captured (by a distance of 26.17) Testing visuals with Needle
  30. CONNECTED PERSONAL OBJECTS 5/2012 Test coverage ‣ Yes, it does

    work too. $ coverage run manage.py test \ myapp.SeleniumTests $ coverage report
  31. CONNECTED PERSONAL OBJECTS 5/2012 Headless mode for continuous integration ‣

    Install a browser and fonts: $ sudo apt-get install -y firefox xfonts-100dpi xfonts-75dpi xfonts- scalable xfonts-cyrillic
  32. CONNECTED PERSONAL OBJECTS 5/2012 Headless mode for continuous integration ‣

    Install a browser and fonts: $ sudo apt-get install -y firefox xfonts-100dpi xfonts-75dpi xfonts- scalable xfonts-cyrillic ‣ Install Xvfb: $ sudo apt-get install -y xvfb
  33. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command:

    $ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test Headless mode for continuous integration
  34. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command:

    $ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test ‣ ... or with Jenkins: Xvfb plugin. Headless mode for continuous integration
  35. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command:

    $ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test ‣ ... or with Jenkins: Xvfb plugin. ‣ ... or with Travis: before_install: - "export DISPLAY=:99.0" - "sh -e /etc/init.d/xvfb start" Headless mode for continuous integration
  36. CONNECTED PERSONAL OBJECTS 5/2012 ‣ ... or with PyVirtualDisplay: from

    pyvirtualdisplay import Display class MyTests(LiveServerTestCase) def setUp(self): self.display = Display( 'xvfb', visible=1,size=(1280, 1024)) self.display.start() def tearDown(self): self.display.stop() Headless mode for continuous integration
  37. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Runs remotely in the cloud

    (no local browser needed). ‣ Gives access to multiple browsers (IE, FF, Opera, Chrome, Safari...) and multiple platforms (Windows, Linux, OSX, Android, iPad, iPhone). Sauce Labs
  38. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Runs remotely in the cloud

    (no local browser needed). ‣ Gives access to multiple browsers (IE, FF, Opera, Chrome, Safari...) and multiple platforms (Windows, Linux, OSX, Android, iPad, iPhone). ‣ Allows tests to be run in parallel. Sauce Labs
  39. CONNECTED PERSONAL OBJECTS 5/2012 Ghost.py from ghost import Ghost class

    MyTests(LiveServerTestCase): def setUp(self): self.ghost = Ghost() super(MyTests, self).setUp() def test_hello(self): self.ghost.open(self.live_server_url + '/hello') self.assertTrue('Hello' in self.ghost.content) ‣ http://jeanphix.me/Ghost.py/
  40. CONNECTED PERSONAL OBJECTS 5/2012 CasperJS from casper.tests import CasperTestCase class

    MyTests(CasperTestCase): def test_something(self): self.assertTrue(self.casper('test.js')) ‣ http://casperjs.org/ ‣ https://github.com/dobarkod/django-casper
  41. CONNECTED PERSONAL OBJECTS 5/2012 Splinter from splinter import Browser class

    MyTests(LiveServerTestCase): def setUp(self): self.browser = Browser() super(MyTests, self).setUp() def test_hello(self): self.browser.visit(self.live_server_url + '/hello') self.assertTrue( self.browser.is_text_present('Hello')) ‣ http://splinter.cobrateam.info
  42. CONNECTED PERSONAL OBJECTS 5/2012 A word of caution... ‣ Selenium

    tests can be a little flaky. ‣ Integration & functional tests are slow. Use them with moderation.
  43. CONNECTED PERSONAL OBJECTS 5/2012 A word of caution... ‣ Selenium

    tests can be a little flaky. ‣ Integration & functional tests are slow. Use them with moderation. ‣ Use LiveServerTestCase or Selenium only for what the dummy client cannot already achieve.
  44. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage.
  45. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend.
  46. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend. ‣ Ensure the user interface works.
  47. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend. ‣ Ensure the user interface works. ‣ Have fun in the process!
  48. CONNECTED PERSONAL OBJECTS 5/2012 Photo credits ‣ http://www.flickr.com/photos/t0msk/3148160756/ ‣ http://www.flickr.com/photos/loozrboy/7311771718/

    ‣ http://www.flickr.com/photos/johnonolan/5836211096/ ‣ http://www.flickr.com/photos/randomskk/3057595640/