Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
2017 - Reverb Chu - Concurrent Testing with py....
Search
PyBay
August 13, 2017
Programming
0
140
2017 - Reverb Chu - Concurrent Testing with py.test
Introducing a different way to perform concurrent testing with the py.test framework.
PyBay
August 13, 2017
Tweet
Share
More Decks by PyBay
See All by PyBay
2017 - The Packaging Gradient
pybay
2
830
2017 - Building Bridges: Stopping Python 2 without damages
pybay
0
590
2017 - Bringing Python 3 to LinkedIn
pybay
1
510
2017 - Python Debugging with PUDB
pybay
0
580
2017 - Opening up to Open Source
pybay
0
220
2017 - A Gentle Introduction to Text Classification with Deep Learning
pybay
2
160
2017 - Performant Asynchronous Programming at Quora
pybay
1
330
2017 - latus - a Personal Cloud Storage App written in Python
pybay
2
450
2017 - Everything You Ever Wanted to Know About Web Authentication in Python
pybay
3
510
Other Decks in Programming
See All in Programming
ピラミッド、アイスクリームコーン、SMURF: 自動テストの最適バランスを求めて / Pyramid Ice-Cream-Cone and SMURF
twada
PRO
9
1k
弊社の「意識チョット低いアーキテクチャ」10選
texmeijin
5
23k
とにかくAWS GameDay!AWSは世界の共通言語! / Anyway, AWS GameDay! AWS is the world's lingua franca!
seike460
PRO
1
550
シールドクラスをはじめよう / Getting Started with Sealed Classes
mackey0225
3
400
Vaporモードを大規模サービスに最速導入して学びを共有する
kazukishimamoto
4
4.3k
外部システム連携先が10を超えるシステムでのアーキテクチャ設計・実装事例
kiwasaki
1
230
2万ページのSSG運用における工夫と注意点 / Vue Fes Japan 2024
chinen
3
1.3k
/←このスケジュール表に立ち向かう フロントエンド開発戦略 / A front-end development strategy to tackle a single-slash schedule.
nrslib
1
590
C#/.NETのこれまでのふりかえり
tomokusaba
1
160
レガシーな Android アプリのリアーキテクチャ戦略
oidy
1
170
Server Driven Compose With Firebase
skydoves
0
400
Modern Angular: Renovation for Your Applications
manfredsteyer
PRO
0
210
Featured
See All Featured
Put a Button on it: Removing Barriers to Going Fast.
kastner
59
3.5k
Thoughts on Productivity
jonyablonski
67
4.3k
How GitHub (no longer) Works
holman
311
140k
Teambox: Starting and Learning
jrom
132
8.7k
Writing Fast Ruby
sferik
626
61k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
43
6.6k
Faster Mobile Websites
deanohume
304
30k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
226
22k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
3
370
Designing the Hi-DPI Web
ddemaree
280
34k
Large-scale JavaScript Application Architecture
addyosmani
510
110k
Optimizing for Happiness
mojombo
376
69k
Transcript
Concurrent Testing with py.test Reverb Chu, Sr. SWE @ Trend
Micro James Wang, UGrad CS Student @ Uni. of Washington
https://goo.gl/mxthn3
https://goo.gl/mxthn3 https://www.apple.com/macbook/
https://goo.gl/mxthn3
Current Solutions https://goo.gl/mxthn3
Current Solutions 1. pytest-xdist https://goo.gl/mxthn3
Current Solutions 1. pytest-xdist 2. ...there’s no 2. https://goo.gl/mxthn3
Why Not pytest-xdist? - Designed for test distribution - Only
support multiprocessing - Not able to do test grouping https://goo.gl/mxthn3
pytest-concurrent life is too short for sequential tests https://goo.gl/mxthn3
Different Concurrent Mode mthread - Impl. with threading - For
IO intensive tests mproc - Impl. with multiprocessing - For CPU intensive tests asyncnet - Impl. with gevent - For network IO intensive tests https://goo.gl/mxthn3
Configurable Worker Amount $ pytest --concmode=mthread --concworkers=10 https://goo.gl/mxthn3
Configurable Worker Amount $ pytest --concmode=mthread --concworkers=max https://goo.gl/mxthn3
Testing in Groups import pytest import time @pytest.mark.concgroup(1) @pytest.mark.parametrize('para', [1,
2, 3, 4, 5]) def test_something(para): time.sleep(2) assert para % 2 @pytest.mark.concgroup(1) def test_something_else(): time.sleep(1) assert 1 == 2 @pytest.mark.concgroup(2) @pytest.mark.parametrize('name', ['this', 'is', 'a', 'book']) def test_second_group(name): time.sleep(2) def test_something_last(): time.sleep(2) raise MemoryError @pytest.mark.concgroup(group: int) https://goo.gl/mxthn3
May ?? I want it... https://goo.gl/mxthn3
May ?? I want it... Jun 20 Intern Onboard https://goo.gl/mxthn3
May ?? I want it... Jul 31 Let’s Do This
Jun 20 Intern Onboard https://goo.gl/mxthn3
May ?? I want it... Aug 11 v0.1.2 Jul 31
Let’s Do This Jun 20 Intern Onboard Aug 10 v0.1.1 Aug 9 v0.1.0 https://goo.gl/mxthn3
$ pip install pytest-concurrent https://goo.gl/mxthn3 @reverbc @jamesw96