BWS 2018: Testing Against Time in JavaScript Applications

As the applications we write tests against grow more complex we might stumble into testing errors stemming from asynchronous and time-dependent behaviour. This talk will characterize several roadblocks in testing which are affected by time. With an example Ember.js application, we will explore how Ember's rich testing API, newest JavaScript features and community libraries, will help us writing legible and reliable tests and how we can make those tests turn green even as time passes by.

30min talk given at the Bulgaria Web Summit 2018.

Transcript

1. Testing against time in JavaScript Applications BWS 2018 @ J J O R D A N _ D E V J E S S I C A J O R D A N
3. https://the-emberjs-times.ongoodbits.com/
4. JavaScript Apps are full of asynchronous behaviour Events & User Interaction Timeouts Ajax Requests …and many more
5. Testing that can be even harder
6. If we get complacent though…
7. …so let’s find out how to do it.
8. Sidetrack Testing in Ember Apps 101
9. + +
10. import { module, test } from 'qunit'; module('some-thing', function(hooks) { test('it computes foo', function(assert) { }); });
11. import { module, test } from 'qunit'; module('some-thing', function(hooks) { hooks.beforeEach(function(){ // custom setup here }); hooks.afterEach(function(){ // custom teardown here }); // test(‘test1', …); // test(‘test2’, …); // test(‘test3’,…); });
12. import { module, test } from 'qunit'; import { setupTest } from 'ember-qunit'; module('some-thing', function(hooks) { setupTest(hooks); test('it computes computedFoo', async function(assert) { }); });
13. import { module, test } from 'qunit'; import { setupTest } from 'ember-qunit'; module('some-thing', function(hooks) { setupTest(hooks); test('it computes computedFoo', async function(assert) { const someThing = this.owner.lookup('service:some-thing'); }); });
14. import { module, test } from 'qunit'; import { setupTest } from 'ember-qunit'; module('some-thing', function(hooks) { setupTest(hooks); test('it computes computedFoo', async function(assert) { const someThing = this.owner.lookup('service:some-thing'); someThing.set('foo', 'baz'); assert.equal(someThing.get('computedFoo'), 'computed baz'); }); });
15. Making Tests Wait
 Async Ops
16. Why is it async? Routing Loading data for a view Handling route transitions Provide context for a view
17. Why is it async? Routing Loading data for a view Handling route transitions Provide context for a view
18. ROUTING Transition States
19. export default Route.extend({ model() { // . . . }, });
20. export default Route.extend({ model() { return $.getJSON( 'https://api.github.com/repos/emberjs/website/pulls' ).then((response) => { return response; }); }, });
21. Ember Data Github Addon Documentation: https://github.com/elwayman02/ember-data-github#ember-data-github Loading Records in Ember Applications via Ember-Data: https://guides.emberjs.com/v3.0.0/routing/specifying-a-routes-model/ export default Route.extend({ model() { const store = this.get('store'); return store.query('github-pull', { repo: 'emberjs/website' }); }, });
22. test('the main page displays a list of PRs', function(assert) { ourOwnVisitHelper('main'); });
23. test('the main page displays a list of PRs', function(assert) { ourOwnVisitHelper('main'); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
24. High Level DOM Assertions With Qunit Dom: https://github.com/simplabs/qunit-dom test('the main page displays a list of PRs', function(assert) { ourOwnVisitHelper('main'); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); }); DOM Assertions with QUnit Dom
25. ❌ test('the main page displays a list of PRs', function(assert) { ourOwnVisitHelper('main'); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
26. Application state takes time to settle
27. Application state takes time to settle pending async operation
28. Application state takes time to settle pending async operation
29. Application state takes time to settle Async operation finishes
30. Application state takes time to settle Async operation finishes settled app state
31. Routing states take time to settle
32. Routing states take time to settle starting Route transition
33. Routing states take time to settle starting Route transition pending HTTP Request
34. Routing states take time to settle starting Route transition pending HTTP Request
35. Routing states take time to settle starting Route transition Promise resolves
36. Routing states take time to settle starting Route transition Promise resolves Route is loaded
37. test('the main page displays a list of PRs', function(assert) { ourOwnVisitHelper('main'); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
38. starting Route transition
39. starting Route transition pending HTTP Request
40. starting Route transition pending HTTP Request
41. starting Route transition pending HTTP Request assert.dom(‘.list-item') .hasText('WIP: Fix Tomster Logo');
42. starting Route transition pending HTTP Request assert.dom(‘.list-item') .hasText('WIP: Fix Tomster Logo'); ❌
43. export default Route.extend({ model() { return $.getJSON( 'https://api.github.com/repos/emberjs/website/pulls' ).then((response) => { return response; }); }, });
44. export default Route.extend({ model() { return $.getJSON( 'https://api.github.com/repos/emberjs/website/pulls' ).then((response) => { return response; }); }, }); once model hook resolves, route transition can progress
45. test('the main page displays a list of PRs', function(assert) { ourOwnVisitHelper('main'); await timeout(300); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
46. starting Route transition
47. starting Route transition pending HTTP Request
48. starting Route transition pending HTTP Request
49. starting Route transition pending HTTP Request await timeout(300);
50. starting Route transition Promise resolves await timeout(300);
51. starting Route transition Promise resolves Route is loaded await timeout(300);
52. starting Route transition Promise resolves Route is loaded assert.dom(‘.list-item') .hasText('WIP: Fix Tomster Logo'); await timeout(300);
53. starting Route transition Promise resolves Route is loaded assert.dom(‘.list-item') .hasText('WIP: Fix Tomster Logo'); ✅ await timeout(300);
54. Hold on, wait a minute…
55. pending HTTP Request What if the request takes longer?
56. pending HTTP Request What if the request takes longer?
57. pending HTTP Request await timeout(300); assert.dom(‘.list-item') .hasText('WIP: Fix Tomster Logo');
58. pending HTTP Request await timeout(300); assert.dom(‘.list-item') .hasText('WIP: Fix Tomster Logo'); ❌
59. ❌ ✅ test('the main page displays a list of PRs', async function(assert) { ourOwnVisitHelper('main'); await timeout(300); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
60. ✅ but… test('the main page displays a list of PRs', async function(assert) { ourOwnVisitHelper('main'); await timeout(300000000000000000000000000000000000000000); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
61. These slow tests though…
62. @ember/test-helpers Framework Agnostic Test Helpers for Ember Apps: https://github.com/emberjs/ember-test-helpers/blob/master/API.md
63. High Level DOM Assertions With Qunit Dom: https://github.com/simplabs/qunit-dom test('the main page displays a list of PRs', function(assert) { ourOwnVisitHelper('main'); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
64. import { visit } from '@ember/test-helpers'; // ... test('the main page displays a list of PRs’, function(assert) { visit('main'); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
65. import { visit } from '@ember/test-helpers'; // ... test('the main page displays a list of PRs', async function(assert) { await visit('main'); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
66. ✅ import { visit } from '@ember/test-helpers'; // ... test('the main page displays a list of PRs', async function(assert) { await visit('main'); assert.dom('.list-item').hasText('WIP: Fix Tomster Logo'); });
67. …will be waiting for async behaviour to settle Helpers for DOM Interaction & more… visit() click() fillIn() keyEvent() triggerEvent() … export default function visit(url){ //... return settled(); }
68. se ttl ed function settled( ) { } hasRunLoop? hasPendingTime rs? hasPendingWait ers? hasPendingReq uests?
69. function settled( ) { } hasRunLoop? hasPendingTime rs? hasPendingWait ers? hasPendingReq uests? every 10 ms se ttl ed
70. function settled( ) { } hasRunLoop? hasPendingTime rs? hasPendingWait ers? hasPendingReq uests? se ttl ed ✅ ✅ ✅ ✅ resolved Promise
71. Writing your own waiting helper See also waitUntil helper from @ember/test-helpers function myOwnWaitHelper(callback) { return new Promise(function(resolve, reject) { let time = 0; // you could of course also have a timer that tracks when this should time out and just reject function checkAssertion(currentTime) { setTImeout(function() { let value = callback(); time += 10; if (value) { // resolve if assertion is correct resolve(value); } else if (timerIsNotUpYet) { checkAssertion(time); } else { reject(value); } }, 10); } checkAssertion(0); }); }
72. Writing your own waiting helper See also waitUntil helper from @ember/test-helpers function myOwnWaitHelper(callback) { return new Promise(function(resolve, reject) { let time = 0; // you could of course also have a timer that tracks when this should time out and just reject function checkAssertion(currentTime) { setTImeout(function() { let value = callback(); time += 10; if (value) { // resolve if assertion is correct resolve(value); } else if (timerIsNotUpYet) { checkAssertion(time); } else { reject(value); } }, 10); } checkAssertion(0); }); }
73. Writing your own waiting helper See also waitUntil helper from @ember/test-helpers function myOwnWaitHelper(callback) { return new Promise(function(resolve, reject) { let time = 0; // you could of course also have a timer that tracks when this should time out and just reject function checkAssertion(currentTime) { setTImeout(function() { let value = callback(); time += 10; if (value) { // resolve if assertion is correct resolve(value); } else if (timerIsNotUpYet) { checkAssertion(time); } else { reject(value); } }, 10); } checkAssertion(0); }); }
74. ✨Testing Magic ✨
75. Async? User Interacti on events might trigger other async operations
76. app/components/pull-request-item.js loadComments(pull) { const commentsUrl = pull.get('commentsUrl'); return this.get('request') .fetch(commentsUrl) .then((comments) => { this.set('comments', comments); }); },
77. app/components/pull-request-item.js export default Component.extend({ request: inject(), // . . . loadComments(pull) { const commentsUrl = pull.get('commentsUrl'); return this.get('request') .fetch(commentsUrl) .then((comments) => { this.set('comments', comments); }); }, }); making use of a dedicated network Service
78. test('loading comments', async function(assert) { await render(hbs`{{pull-request-item pullRequest=pullRequest}}`); await click('button.load-comments'); assert.dom(‘.list-item .comment').hasText('Comment 1: LGTM!'); });
79. test('loading comments', async function(assert) { await render(hbs`{{pull-request-item pullRequest=pullRequest}}`); await click('button.load-comments'); assert.dom(‘.list-item .comment').hasText('Comment 1: LGTM!'); }); ❌
80. Mocking Requests td.when().thenResolve() td.when().thenReject() td.when().thenCallback()
81. this.owner.register(‘service:request', RequestService.extend({ fetch: td.function(), });
82. test('loading comments', async function(assert) { this.owner.register(‘service:request', RequestService.extend({ fetch: td.function(), }); td.when(this.get('request').fetch(commentsUrl)) .thenResolve(this.mockComments); await render(hbs`{{pull-request-item pullRequest=pullRequest}}`); await click('button.load-comments'); assert.dom('.list-item .comment').hasText('Comment 1: LGTM!'); });
83. ✅ test('loading comments', async function(assert) { this.owner.register(‘service:request', RequestService.extend({ fetch: td.function(), }); td.when(this.get('request').fetch(commentsUrl)) .thenResolve(this.mockComments); await render(hbs`{{pull-request-item pullRequest=pullRequest}}`); await click('button.load-comments'); assert.dom('.list-item .comment').hasText('Comment 1: LGTM!'); });
84. Making tests pass anytime 
 Time Travel
85. test('shows the week day', async function(assert) { await visit('main'); assert.dom('aside.week-day').hasText('Monday'); });
86. works when run on Mondays… ✅ test('shows the week day', async function(assert) { await visit('main'); assert.dom('aside.week-day').hasText('Monday'); });
87. …and fails Tuesday - Sunday ❌ test('shows the week day', async function(assert) { await visit('main'); assert.dom('aside.week-day').hasText('Monday'); });
88. Travel through time with Timecop.js Timecop Ember addon: https://github.com/matteodepalo/ember-cli-timecop Timecop.js: https://github.com/jamesarosen/Timecop.js
89. Timecop.travel(new Date(2017, 9, 11, 11, 45)); // is a Monday
90. hooks.beforeEach(function() Timecop.install(); Timecop.travel(new Date(2017, 9, 11, 11, 45)); // is a Monday }); hooks.afterEach(function(){ Timecop.uninstall(); });
91. ✅ module('Acceptance | overview', function(hooks) { hooks.beforeEach() Timecop.install(); Timecop.travel(new Date(2017, 9, 11, 11, 45)); // is a Monday }, hooks.afterEach(){ Timecop.uninstall(); } test('shows the weekday', async function(assert) { await visit('main'); assert.dom('aside.week-day').hasText('Monday'); }); });
92. Overcome timing challenges in tests Use time travel to test time-dependent code Mock async network requests reliably Wait for application state to settle Make tests pass regardless of execution order …
93. Timing in your tests matters
94. Thank you. @ J J O R D A N _ D E V GITHUB:JESSICA-JORDAN @Bulgaria Web Summit 2018