Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Automation Tests at Babylon
Search
Ilya Puchka
October 03, 2018
Programming
0
92
Automation Tests at Babylon
Ilya Puchka
October 03, 2018
Tweet
Share
More Decks by Ilya Puchka
See All by Ilya Puchka
Dependency Injection in Swift
ilyapuchka
2
1.5k
Other Decks in Programming
See All in Programming
CSC509 Lecture 03
javiergs
PRO
0
330
Railsだからできる 例外業務に禍根を残さない 設定設計パターン
ei_ei_eiichi
0
160
Pull-Requestの内容を1クリックで動作確認可能にするワークフロー
natmark
2
450
ネイティブ製ガントチャートUIを作って学ぶUICollectionViewLayoutの威力
jrsaruo
0
130
Reduxモダナイズ 〜コードのモダン化を通して、将来のライブラリ移行に備える〜
pvcresin
2
680
Playwrightはどのようにクロスブラウザをサポートしているのか
yotahada3
7
2.3k
AI Coding Meetup #3 - 導入セッション / ai-coding-meetup-3
izumin5210
0
420
LLMとPlaywright/reg-suitを活用した jQueryリファクタリングの実際
kinocoboy2
4
660
Local Peer-to-Peer APIはどのように使われていくのか?
hal_spidernight
2
440
AIで開発生産性を上げる個人とチームの取り組み
taniigo
0
130
2分台で1500examples完走!爆速CIを支える環境構築術 - Kaigi on Rails 2025
falcon8823
3
3k
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
110
Featured
See All Featured
Thoughts on Productivity
jonyablonski
70
4.9k
A Modern Web Designer's Workflow
chriscoyier
697
190k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
960
Fireside Chat
paigeccino
40
3.7k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.6k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
6.1k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
610
Visualization
eitanlees
148
16k
Transcript
Automation Tests at Babylon
What we will talk about: — What issues do we
face in UI tests — How do we solve them — BDD-style UI tests — Tooling
What we will not talk about: — Why — How
— What to test
What issues do we face? — unexpected failures — performance
— scenarios hard to automate & hard to test manually
Content stability
External system actions required — start/complete appointment — subscribe prescription
— answer patient question — etc
Network stability* * https://techcrunch.com/2016/07/13/facebook-lifts- the-veil-on-its-mobile-device-lab
Performance Repeated test steps — register new user — open
Appointments tab — enter your symptoms — book an appointment — add payment method — confirm payment
large tests, faster test suite vs focused tests, slower test
suite
How do we solve that?
Control the World** — Control the network — Control the
initial state of the app ** C. Montgomery Burns' Handbook of World Domination By Matt Groening.
How to control the network control == own data
How to control the network — from inside the app
— from inside the test runner
From inside the app - in-app stubs — straightforward ✅
— integrates in application code — not very flexible ❌
From inside test runner - local web server — harder
to setup ❌ — integrates in test runner ✅ — very flexible ✅
Local server setup In the app: — switch environment
Local server setup In the app: — allow local networking
in the app
Local server setup In the test: — web server implementation
(Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
Local server setup In the test: — web server implementation
(Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
Local server edge cases — Parallel testing — Absolute urls
in responses — Overriding response size
Local web server use cases — Logging — Snapshots —
Full mocks
Network session recording — Record and replay using mock URLSession
(Vinyl, DVR) — detect recording mode — configure matching strategy — create URLSession mock
Network session recording — Record random test input — match
full request — record random body data in a file
Network session recording — Override current date
Mocked backend — mock everything ❌ — a lot of
boilerplate code to write — total control ✅
Network stability Content stability ✅
Tests performance
Tests performance Shortcuts via launch arguments — Special feature toggles
— User defaults overrides — Memory storage
Tests performance API calls from tests — to register a
new user — to add a family member — to book an appointment — etc
BDD-style tests
None
None
Tests building blocks — Page Objects — Step Definers —
Features (XCTestCase)
None
Links Web servers envoy/Ambassador httpswift/swifter swisspol/GCDWebServer Snapshots Velhotes/Vinyl venmo/DVR BDD
tests net-a-porter-mobile/XCTest-Gherkin Ahmed-Ali/Cucumberish Other tools TitouanVanBelle/XCTestHTMLReport