Integration and Functional Testing with LiveServerTestCase, Selenium and More.

~ Julien Phalip Integration and Functional Testing with LiveServerTestCase, Selenium
and More. September 5th, 2013 DJANGOCON US 2013

CONNECTED PERSONAL OBJECTS 5/2012 About me... ‣ Worked with Django
since 2007 (version 0.96). ‣ Django core committer since 2011.

CONNECTED PERSONAL OBJECTS 5/2012 www.nurun.com

CONNECTED PERSONAL OBJECTS 5/2012 Definitions (subject to debate)

CONNECTED PERSONAL OBJECTS 5/2012 Definitions (subject to debate) ‣ Unit
testing Ensure that small parts work in isolation.

testing Ensure that small parts work in isolation. ‣ Integration testing Ensure that those parts work well together.

testing Ensure that small parts work in isolation. ‣ Integration testing Ensure that those parts work well together. ‣ Functional testing Ensure that the application works, from a user’s perspective.

CONNECTED PERSONAL OBJECTS 5/2012 Testing in Django (the traditional way)

CONNECTED PERSONAL OBJECTS 5/2012 Testing in Django (the traditional way)
from django.test import TestCase class MyTests(TestCase): def test_hello(self): r = self.client.get('/hello') self.assertTrue('Hello' in r.content)

CONNECTED PERSONAL OBJECTS 5/2012 django.test.TestCase

CONNECTED PERSONAL OBJECTS 5/2012 django.test.TestCase DJANGO THREAD

CONNECTED PERSONAL OBJECTS 5/2012 django.test.TestCase Application code DJANGO THREAD

CONNECTED PERSONAL OBJECTS 5/2012 django.test.TestCase TestCase Application code DJANGO THREAD

CONNECTED PERSONAL OBJECTS 5/2012 django.test.TestCase Database TestCase Application code DJANGO
THREAD

CONNECTED PERSONAL OBJECTS 5/2012 django.test.TestCase “Dummy” http client (self.client) Database
TestCase Application code DJANGO THREAD

TestCase Application code MockRequest DJANGO THREAD

TestCase Application code MockRequest HttpResponse DJANGO THREAD

CONNECTED PERSONAL OBJECTS 5/2012 Pretty good but incomplete...

CONNECTED PERSONAL OBJECTS 5/2012 Pretty good but incomplete... ‣ Does
not exercise the full http protocol.

CONNECTED PERSONAL OBJECTS 5/2012 Pretty good but incomplete... ‣ Does
not exercise the full http protocol. ‣ Does not allow to test the user interface.

And then came LiveServerTestCase...

CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase

CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase LiveServerTestCase DJANGO THREAD #1

CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase LiveServerTestCase Application code DJANGO THREAD
#2 DJANGO THREAD #1 Live http server

CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client LiveServerTestCase Application
code DJANGO THREAD #2 DJANGO THREAD #1 Live http server

CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase
Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server

http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server

http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi

CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase Database LiveServerTestCase http python-requests Application
code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi

CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase Database LiveServerTestCase http Application code
DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi

CONNECTED PERSONAL OBJECTS 5/2012 Some important details ‣ Live server
listens to localhost:8081 by default ‣ Address can be overridden: ./manage.py test --liveserver=localhost:8082 ./manage.py test --liveserver=localhost:9000-9200 ‣ Address accessible from the test’s code: self.live_server_url

CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests

CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests ‣ Testing REST
APIs:

CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests ‣ Testing REST
APIs: import requests class MyTests(LiveServerTestCase): def test_api_user(self): r = requests.get( self.live_server_url + '/api/user/1/') self.assertEqual(r.status_code, 200) self.assertEqual(r.json, {'username': 'john.doe', 'email': '[email protected]'})

CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests

CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests ‣ Testing OAUTH
workflows:

CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests ‣ Testing OAUTH
workflows: import requests from requests_oauthlib import OAuth2 from oauthlib.oauth2 import BackendApplicationClient class MyTests(LiveServerTestCase): def test_oauth(self): backend = BackendApplicationClient('mybackend') oauth2 = OAuth2(client=backend, token=TOKEN) r = requests.get( self.live_server_url + '/oauth', auth=oauth2) self.assertEqual(r.status_code, 200) self.assertTrue('Hello' in r.content)

Browser automators

CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators?

CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer
to real Web user environment.

to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded.

to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed.

to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested.

to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested. ‣ Browser compatibility can be tested.

to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested. ‣ Browser compatibility can be tested. ‣ It’s fun!

CONNECTED PERSONAL OBJECTS 5/2012 Selenium

CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ API to become W3C
standard (currently in draft)

standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python.

standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python. ‣ Works across multiple browsers (Firefox, Chrome, IE, Safari, Opera)

standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python. ‣ Works across multiple browsers (Firefox, Chrome, IE, Safari, Opera) ‣ Easy to install: pip install selenium

CONNECTED PERSONAL OBJECTS 5/2012 from django.test import LiveServerTestCase from selenium.webdriver.firefox.webdriver
import \ WebDriver class MyTests(LiveServerTestCase):

import \ WebDriver class MyTests(LiveServerTestCase): def setUp(self): self.driver = WebDriver() super(MyTests, self).setUp() def tearDown(self): self.driver.quit() super(MyTests, self).tearDown()

import \ WebDriver class MyTests(LiveServerTestCase): ...

import \ WebDriver class MyTests(LiveServerTestCase): def test_hello(self): self.driver.get( self.live_server_url + '/hello') e = self.driver.find_element_by_id('btn') e.click() ... ...

CONNECTED PERSONAL OBJECTS 5/2012 DEMO Django Core Test Suite

Tips and goodies

CONNECTED PERSONAL OBJECTS 5/2012 Testing visuals with Needle

CONNECTED PERSONAL OBJECTS 5/2012 Testing visuals with Needle ‣ https://github.com/bfirsh/needle

‣ Based on Selenium.

‣ Based on Selenium. ‣ Tests visuals by comparing screenshots.

CONNECTED PERSONAL OBJECTS 5/2012 Testing visuals with Needle ‣ Test
the looks of an entire page...

CONNECTED PERSONAL OBJECTS 5/2012 Testing visuals with Needle ‣ Or
just parts of it.

CONNECTED PERSONAL OBJECTS 5/2012 from needle.cases import NeedleTestCase from from
selenium.webdriver import firefox class MyTests(NeedleTestCase, LiveServerTestCase): driver_command_executor = firefox.ExtensionConnection( "127.0.0.1", firefox.FirefoxProfile()) def test_homepage_menu(self): self.driver.get(self.live_server_url) self.assertScreenshot('ul.menu', 'menu') Testing visuals with Needle

CONNECTED PERSONAL OBJECTS 5/2012 ====================================================================== FAIL: test_homepage_menu (myapp.tests.MyTests) ---------------------------------------------------------------------- Traceback
(most recent call last): File "myapp/tests.py", line 115, in test_homepage_menu self.assertScreenshot('ul.menu', 'menu') File "/.virtualenvs/myproject/src/needle/needle/cases.py", line 99, in assertScreenshot pass File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/ python2.6/contextlib.py", line 23, in __exit__ self.gen.next() File "/.virtualenvs/myproject/src/needle/needle/cases.py", line 160, in compareScreenshot % (filename, distance)) AssertionError: The saved screenshot for 'menu' did not match the screenshot captured (by a distance of 26.17) Testing visuals with Needle

CONNECTED PERSONAL OBJECTS 5/2012 Test coverage

CONNECTED PERSONAL OBJECTS 5/2012 Test coverage ‣ Yes, it does
work too.

CONNECTED PERSONAL OBJECTS 5/2012 Test coverage ‣ Yes, it does
work too. $ coverage run manage.py test \ myapp.SeleniumTests $ coverage report

CONNECTED PERSONAL OBJECTS 5/2012 Headless mode for continuous integration

CONNECTED PERSONAL OBJECTS 5/2012 Headless mode for continuous integration ‣
Install a browser and fonts: $ sudo apt-get install -y firefox xfonts-100dpi xfonts-75dpi xfonts- scalable xfonts-cyrillic

CONNECTED PERSONAL OBJECTS 5/2012 Headless mode for continuous integration ‣
Install a browser and fonts: $ sudo apt-get install -y firefox xfonts-100dpi xfonts-75dpi xfonts- scalable xfonts-cyrillic ‣ Install Xvfb: $ sudo apt-get install -y xvfb

CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command:
$ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test Headless mode for continuous integration

$ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test ‣ ... or with Jenkins: Xvfb plugin. Headless mode for continuous integration

$ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test ‣ ... or with Jenkins: Xvfb plugin. ‣ ... or with Travis: before_install: - "export DISPLAY=:99.0" - "sh -e /etc/init.d/xvfb start" Headless mode for continuous integration

CONNECTED PERSONAL OBJECTS 5/2012 ‣ ... or with PyVirtualDisplay: from
pyvirtualdisplay import Display class MyTests(LiveServerTestCase) def setUp(self): self.display = Display( 'xvfb', visible=1,size=(1280, 1024)) self.display.start() def tearDown(self): self.display.stop() Headless mode for continuous integration

CONNECTED PERSONAL OBJECTS 5/2012 Sauce Labs

CONNECTED PERSONAL OBJECTS 5/2012 ‣ Runs remotely in the cloud
(no local browser needed). Sauce Labs

(no local browser needed). ‣ Gives access to multiple browsers (IE, FF, Opera, Chrome, Safari...) and multiple platforms (Windows, Linux, OSX, Android, iPad, iPhone). Sauce Labs

(no local browser needed). ‣ Gives access to multiple browsers (IE, FF, Opera, Chrome, Safari...) and multiple platforms (Windows, Linux, OSX, Android, iPad, iPhone). ‣ Allows tests to be run in parallel. Sauce Labs

CONNECTED PERSONAL OBJECTS 5/2012

Not just Selenium...

CONNECTED PERSONAL OBJECTS 5/2012 Ghost.py ‣ http://jeanphix.me/Ghost.py/

CONNECTED PERSONAL OBJECTS 5/2012 Ghost.py from ghost import Ghost class
MyTests(LiveServerTestCase): def setUp(self): self.ghost = Ghost() super(MyTests, self).setUp() def test_hello(self): self.ghost.open(self.live_server_url + '/hello') self.assertTrue('Hello' in self.ghost.content) ‣ http://jeanphix.me/Ghost.py/

CONNECTED PERSONAL OBJECTS 5/2012 CasperJS ‣ http://casperjs.org/ ‣ https://github.com/dobarkod/django-casper

CONNECTED PERSONAL OBJECTS 5/2012 CasperJS from casper.tests import CasperTestCase class
MyTests(CasperTestCase): def test_something(self): self.assertTrue(self.casper('test.js')) ‣ http://casperjs.org/ ‣ https://github.com/dobarkod/django-casper

CONNECTED PERSONAL OBJECTS 5/2012 Splinter ‣ http://splinter.cobrateam.info

CONNECTED PERSONAL OBJECTS 5/2012 Splinter from splinter import Browser class
MyTests(LiveServerTestCase): def setUp(self): self.browser = Browser() super(MyTests, self).setUp() def test_hello(self): self.browser.visit(self.live_server_url + '/hello') self.assertTrue( self.browser.is_text_present('Hello')) ‣ http://splinter.cobrateam.info

Epilogue...

CONNECTED PERSONAL OBJECTS 5/2012 A word of caution...

CONNECTED PERSONAL OBJECTS 5/2012 A word of caution... ‣ Selenium
tests can be a little flaky.

tests can be a little flaky. ‣ Integration & functional tests are slow. Use them with moderation.

tests can be a little flaky. ‣ Integration & functional tests are slow. Use them with moderation. ‣ Use LiveServerTestCase or Selenium only for what the dummy client cannot already achieve.

CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

‣ Increase your confidence in your code.

‣ Increase your confidence in your code. ‣ Increase your test coverage.

‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend.

‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend. ‣ Ensure the user interface works.

‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend. ‣ Ensure the user interface works. ‣ Have fun in the process!

CONNECTED PERSONAL OBJECTS 5/2012 Photo credits ‣ http://www.flickr.com/photos/t0msk/3148160756/ ‣ http://www.flickr.com/photos/loozrboy/7311771718/
‣ http://www.flickr.com/photos/johnonolan/5836211096/ ‣ http://www.flickr.com/photos/randomskk/3057595640/

@julienphalip http:/ /julienphalip.com http:/ /nurun.com Thank you!

Integration and Functional Testing with LiveSer...

Integration and Functional Testing with LiveServerTestCase, Selenium and More.

More Decks by Julien Phalip

Other Decks in Programming

Featured

Transcript