Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Automation Tests at Babylon
Search
Ilya Puchka
October 03, 2018
Programming
0
91
Automation Tests at Babylon
Ilya Puchka
October 03, 2018
Tweet
Share
More Decks by Ilya Puchka
See All by Ilya Puchka
Dependency Injection in Swift
ilyapuchka
2
1.5k
Other Decks in Programming
See All in Programming
すべてのコンテキストを、 ユーザー価値に変える
applism118
2
470
PHPで始める振る舞い駆動開発(Behaviour-Driven Development)
ohmori_yusuke
2
140
The Evolution of Enterprise Java with Jakarta EE 11 and Beyond
ivargrimstad
1
850
イベントストーミング図からコードへの変換手順 / Procedure for Converting Event Storming Diagrams to Code
nrslib
1
180
地方に住むエンジニアの残酷な現実とキャリア論
ichimichi
2
640
データベースコネクションプール(DBCP)の変遷と理解
fujikawa8
1
270
Kotlin エンジニアへ送る:Swift 案件に参加させられる日に備えて~似てるけど色々違う Swift の仕様 / from Kotlin to Swift
lovee
1
250
無関心の谷
kanayannet
0
180
XSLTで作るBrainfuck処理系
makki_d
0
210
Railsアプリケーションと パフォーマンスチューニング ー 秒間5万リクエストの モバイルオーダーシステムを支える事例 ー Rubyセミナー 大阪
falcon8823
3
790
GoのWebAssembly活用パターン紹介
syumai
3
10k
Haskell でアルゴリズムを抽象化する / 関数型言語で競技プログラミング
naoya
17
4.8k
Featured
See All Featured
Navigating Team Friction
lara
187
15k
BBQ
matthewcrist
89
9.7k
Testing 201, or: Great Expectations
jmmastey
42
7.5k
Writing Fast Ruby
sferik
628
61k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.8k
A Modern Web Designer's Workflow
chriscoyier
693
190k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
20
1.3k
Optimising Largest Contentful Paint
csswizardry
37
3.3k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Producing Creativity
orderedlist
PRO
346
40k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Thoughts on Productivity
jonyablonski
69
4.7k
Transcript
Automation Tests at Babylon
What we will talk about: — What issues do we
face in UI tests — How do we solve them — BDD-style UI tests — Tooling
What we will not talk about: — Why — How
— What to test
What issues do we face? — unexpected failures — performance
— scenarios hard to automate & hard to test manually
Content stability
External system actions required — start/complete appointment — subscribe prescription
— answer patient question — etc
Network stability* * https://techcrunch.com/2016/07/13/facebook-lifts- the-veil-on-its-mobile-device-lab
Performance Repeated test steps — register new user — open
Appointments tab — enter your symptoms — book an appointment — add payment method — confirm payment
large tests, faster test suite vs focused tests, slower test
suite
How do we solve that?
Control the World** — Control the network — Control the
initial state of the app ** C. Montgomery Burns' Handbook of World Domination By Matt Groening.
How to control the network control == own data
How to control the network — from inside the app
— from inside the test runner
From inside the app - in-app stubs — straightforward ✅
— integrates in application code — not very flexible ❌
From inside test runner - local web server — harder
to setup ❌ — integrates in test runner ✅ — very flexible ✅
Local server setup In the app: — switch environment
Local server setup In the app: — allow local networking
in the app
Local server setup In the test: — web server implementation
(Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
Local server setup In the test: — web server implementation
(Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
Local server edge cases — Parallel testing — Absolute urls
in responses — Overriding response size
Local web server use cases — Logging — Snapshots —
Full mocks
Network session recording — Record and replay using mock URLSession
(Vinyl, DVR) — detect recording mode — configure matching strategy — create URLSession mock
Network session recording — Record random test input — match
full request — record random body data in a file
Network session recording — Override current date
Mocked backend — mock everything ❌ — a lot of
boilerplate code to write — total control ✅
Network stability Content stability ✅
Tests performance
Tests performance Shortcuts via launch arguments — Special feature toggles
— User defaults overrides — Memory storage
Tests performance API calls from tests — to register a
new user — to add a family member — to book an appointment — etc
BDD-style tests
None
None
Tests building blocks — Page Objects — Step Definers —
Features (XCTestCase)
None
Links Web servers envoy/Ambassador httpswift/swifter swisspol/GCDWebServer Snapshots Velhotes/Vinyl venmo/DVR BDD
tests net-a-porter-mobile/XCTest-Gherkin Ahmed-Ali/Cucumberish Other tools TitouanVanBelle/XCTestHTMLReport