Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Jetpack Benchmark - Droidcon RO 2019

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for JB Lorenzo JB Lorenzo
October 04, 2019
130
0

Jetpack Benchmark - Droidcon RO 2019

Avatar for JB Lorenzo

JB Lorenzo

October 04, 2019

More Decks by JB Lorenzo

See All by JB Lorenzo
Measure What Matters with Jetpack Benchmark
Avatar for JB Lorenzo bennegeek
0
200
Lessons Learned from Using Kotlin Multiplatform internally
Avatar for JB Lorenzo bennegeek
0
140
Measure What Matters with Jetpack Benchmark
Avatar for JB Lorenzo bennegeek
0
95
Droidkaigi 2019 flutter kotlin native
Avatar for JB Lorenzo bennegeek
2
620
Fast Prototypes using Kotlin_Native (and Flutter)
Avatar for JB Lorenzo bennegeek
1
91
kotlin-native-workshop.pdf
Avatar for JB Lorenzo bennegeek
0
130
Fast Prototypes with Flutter + Kotlin/Native
Avatar for JB Lorenzo bennegeek
0
590

Featured

See All Featured
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1369
210k
The Web Performance Landscape in 2024 [PerfNow 2024]
Avatar for Tammy Everts tammyeverts
12
1.3k
Leveraging Curiosity to Care for An Aging Population
Avatar for Cassini Nazir cassininazir
1
520
brightonSEO & MeasureFest 2025 - Winning Strategies for Black Friday CRO & PPC - Christian Goodrich
Avatar for Christian Goodrich cargoodrich
3
860
How to Grow Your eCommerce with AI & Automation
Avatar for Katarina Dahlin katarinadahlin
PRO
2
310
Understanding Cognitive Biases in Performance Measurement
Avatar for Philip Tellis bluesmoon
32
3k
The SEO Collaboration Effect
Avatar for Kristina Bergwall kristinabergwall1
1
590
AI: The stuff that nobody shows you
Avatar for John Nunemaker jnunemaker
PRO
10
1.1k
AI Search:
Where Are We & What Can We Do About It?
Avatar for Aleyda Solis aleyda
0
8k
Getting science done with accelerated Python computing platforms
Avatar for Jacob Tomlinson jacobtomlinson
2
520
Amusing Abliteration
Avatar for ianozsvald ianozsvald
1
330
State of Search Keynote: SEO is Dead Long Live SEO
Avatar for Ryan Jones ryanjones
0
300

Transcript

  1. @bennegeek Measure What Matters with Jetpack Benchmark JB Lorenzo |

    #DroidconRO19
  2. None

  3. @bennegeek

  4. @bennegeek

  5. @bennegeek

  6. @bennegeek Benchmark 1.0.0-alpha06 September 18, 2019 alpha01 on May 7

  7. @bennegeek From alpha02 to alpha06 * Auto disables test coverage

    * AndroidBenchmarkRunner by default * Module structure changed to decouple JUnit4 * Change in package name * Documentation for CI integration

  8. @bennegeek Documentation for CI Integration https://developer.android.com/studio/profile/run-benchmarks-in-ci * Guidelines for benchmark

    without Gradle * Self-instrumented APK * Separate app and test APKs * Guidelines for CI * Important flags (e.g. lock clocks, json output)

  9. @bennegeek Setting up Android Studio 3.5 Beta npw.benchmark.template.module=true

  10. @bennegeek Setting up

  11. @bennegeek Setting up your.package.name.

  12. @bennegeek Setting up dependencies { androidTestImplementation “androidx.benchmark:benchmark:1.0.0-alpha06" }

  13. @bennegeek Setting up <application android:debuggable="false" tools:ignore="HardcodedDebugMode" tools:replace="android:debuggable"/>

  14. @bennegeek Let’s measure!

  15. @bennegeek @mootazltaief

  16. @bennegeek @mootazltaief

  17. @bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own

    Device Farm

  18. @bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own

    Device Farm

  19. @bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own

    Device Farm

  20. @bennegeek Amazon EC2 + Emulator AWS: * Does not allow

    nested VM * Need to run bare-Metal instance to run emulator Emulator: * Not advised for benchmarks Other cloud providers: * has support for nested VM

  21. @bennegeek Integrating in Gitlab CI // benchmark/build.gradle apply plugin: androidx.benchmark

    // if device/emulator is connected to CI > ./gradlew benchmark:connectedCheck

  22. @bennegeek Integrating in Gitlab CI { "context": { "build": {

    "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,

  23. @bennegeek Integrating in Gitlab CI { "context": { "build": {

    "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,

  24. @bennegeek Integrating in Gitlab CI { "context": { "build": {

    "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,

  25. @bennegeek Integrating in Gitlab CI { "context": { "build": {

    "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500, 95250, ...

  26. @bennegeek Integrating in Gitlab CI { "context": { "build": {

    "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,

  27. @bennegeek Benchmark Warnings

  28. @bennegeek Benchmark Warnings

  29. @bennegeek Benchmark Warnings

  30. @bennegeek Benchmark Warnings

  31. @bennegeek Benchmark Warnings

  32. @bennegeek Benchmark Warnings

  33. @bennegeek Benchmark Warnings

  34. @bennegeek Benchmark Warnings

  35. @bennegeek Benchmark Warnings

  36. @bennegeek Benchmark Warnings

  37. @bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own

    Device Farm

  38. @bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own

    Device Farm

  39. @bennegeek Firebase Test Lab Integration with CI Firebase Test Lab

    * Problems when running test with mocked network * $5/device/hour after Free tier * Need to configure gcloud in CI * Fetch results in google cloud storage

  40. @bennegeek Firebase Test Lab Integration with CI Firebase Test Lab

    How to get the json output from test lab?

  41. @bennegeek Gitlab CI

  42. @bennegeek Gitlab CI

  43. @bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c '''

    import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''

  44. @bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c '''

    import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''

  45. @bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c '''

    import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''

  46. @bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c '''

    import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''

  47. @bennegeek Gitlab CI PreviousBenchmark = 138375 Threshold = 0.10 if

    medians["benchmarkJacksonWithSample1"] > (PreviousBenchmark * Threshold): print("benchmarkJacksonWithSample1 is beyond threshold") exit(1)

  48. @bennegeek Output

  49. @bennegeek Output

  50. @bennegeek Output

  51. @bennegeek Next Steps * Separate app into modules * Mock

    more data for benchmarking * Try it out yourself

  52. @bennegeek Next Steps * Separate app into modules * Mock

    more data for benchmarking * Try it out yourself * Visit the issue tracker * File bugs

  53. @bennegeek tech.olx.com Example project (No OLX code): http://github.com/mootazltaief/ android-json-benchmark

  54. @bennegeek tech.olx.com https://issuetracker.google.com/ issues?q=componentid: 585351%20status:open

  55. None

  56. @bennegeek tech.olx.com

  57. None
  58. None

  59. Berlin Buenos Aires Lisbon New Delhi Poznan Dubai

  60. @bennegeek Multumesc! vreo întrebare? Get in touch with us olxgroup.com

    tech.olx.com olxtechberlin [email protected] jblorenzo JB Lorenzo | #DroidconRO19