Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Continuous Integration for Android Using Jenkins

Continuous Integration for Android Using Jenkins

This talk explains how Jenkins can be used to improve product quality, stability and delivery speed by embracing continuous integration. We’ll walk through how Android developers and testers can benefit from using Jenkins, and then give some pro tips for more advanced capabilities, including how to speed up your tests by running them in parallel across multiple Jenkins slaves.

It was presented at the Android Summit 2016: http://androidsummit.org/

VIDEO: https://www.youtube.com/watch?v=wNXJi75WHcA

Dan Jarvis: http://careers.stackoverflow.com/dj
Sam Edwards: https://twitter.com/handstandsam

Sam Edwards

August 26, 2016
Tweet

More Decks by Sam Edwards

Other Decks in Programming

Transcript

  1. 05 About Us – Dan Jarvis & Sam Edwards •

    Tech Leads for Capital One Wallet on Android • Top rated finance app by a US bank • First US bank with native Android contactless payments • Each has 12+ years professional experience • Passionate about automation and knowledge sharing @HandstandSam
  2. • Continuous Integration (CI) fundamentals • Productivity tips • Capital

    One Wallet case study • Jenkins as a platform • Pro tips - how to run tests in parallel 05 Agenda
  3. Continuous Integration (CI) is a development practice that requires developers

    to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early. ”
  4. Why? • Deliver faster and higher quality • Your product

    is always ready to deploy • Capture institutional knowledge in tests • Automation hugely improves productivity • Multiply your impact!
  5. Jenkins Plugins • Plugins can be written to extend Jenkins

    capabilities Default Install Plugins Added
  6. Jenkins Master, Slaves and Executors • Jobs are configured on

    the master node • The master dispatches jobs to run on executors on slave nodes (which could be on the same machine as the master)
  7. What Makes Jenkins so Great? • Easy, installation, configuration &

    a web based interface • Open source https://github.com/jenkinsci/jenkins • >1000 plugins • Distributed and scalable • Long history and widespread adoption • Enterprise integrations
  8. Running Specific Instrumentation Tests (docs) Run By Instrumentation Argument(s) Code

    Method -e class com.FooTest#test1 Class -e class com.FooTest Standard Annotations -e size small -e size medium -e size large @SmallTest @MediumTest @LargeTest Custom Annotations -e annotation -e notAnnotation @MyCustomAnnotation Package –e package
  9. The Testing Pyramid - Wallet Examples • Android Espresso Tests,

    Real Servers • Android Tests, Real Servers • JVM Tests, Mock Data • Android Component Tests, Mock Data Black Box Tests Integration Tests Component Tests Unit Tests
  10. Capital One Wallet for Android on Jenkins Today Git Commit

    Pull Request (PR) Opened or Updated Unit Tests on PR Branch ✓ PR Merged Assemble APKs and Test APKs Integration Tests Unit Tests Component Tests Blackbox Tests ✓ ✓ ✓ ✓ ✓ ✓
  11. Better – Four Jobs Run Component Tests Run Unit Tests

    Compile/Build Compile/Build Compile/Build Run Blackbox Tests Run Integration Tests Compile/Build
  12. Better – Four Jobs Run Component Tests Run Unit Tests

    Compile/Build Compile/Build Compile/Build Run Blackbox Tests Run Integration Tests Compile/Build Limitations: • Not great if you want to repeatedly re-run the tests • Wastes time/money if you have less than four slaves/executors
  13. Better – Five Jobs Run Component Tests Run Unit Tests

    Compile/Build Run Blackbox Tests Run Integration Tests
  14. Better – Five Jobs Run Component Tests Run Unit Tests

    Compile/Build Run Blackbox Tests Run Integration Tests
  15. What if I told you every build that passes your

    automated tests is a release candidate? ”
  16. iOS Stable Integration Tests Server Unit Tests Continuous Delivery for

    Servers ✓ Server Integration Tests Android Stable Integration Tests ✓ ✓ ✓ ✓
  17. Continuous Delivery for Servers 1. Server developer runs unit tests

    locally 2. On pull request, unit tests and all server and client integration tests run on a transient server instance
  18. Continuous Delivery for Servers 1. Server developer runs unit tests

    locally 2. On pull request, unit tests and all server and client integration tests run on a transient server instance 3. On merge, server code gets automatically deployed to shared QA region
  19. Manage Scripts in Separate Repository Benefits: • Versioned with great

    tooling to view history • Great when you have to update multiple or similar scripts like a library version number
  20. Manage Scripts in Separate Repository Benefits: • Versioned with great

    tooling to view history • Great when you have to update multiple or similar scripts like a library version number • Great when re-building Jenkins
  21. Manage Scripts in Separate Repository Benefits: • Versioned with great

    tooling to view history • Great when you have to update multiple or similar scripts like a library version number • Great when re-building Jenkins
  22. Sharding Your Tests Run Component Tests Run Unit Tests Compile/Build

    Run Blackbox Tests Run Integration Tests Blackbox Tests [0] Blackbox Tests [1] Blackbox Tests [2] Blackbox Tests [3]
  23. • The main job uses up an executor while waiting

    for Phases to complete (which each use their own executor) • Phases need to do their own partitioning of the test suite • Test results are not aggregated back into a single report 05 Problems with MultiJob Plugin
  24. • Easier to pass sharding index to build executors •

    Starts parallel jobs for all combinations of configurations 05 Benefits of Matrix Plugin
  25. 05 Android Emulator PROS • Multiple screen sizes are easy

    • FREE! • Fast (10x faster install speed then normal devices) CONS • “Snapshot” feature is unreliable • Workaround: Re-create the AVD every time
  26. 05 Genymotion Emulator PROS • Multiple screen sizes are easy

    • Fast • Snapshot feature works pretty well CONS • Requires Licenses • Can be difficult to manage with sharding
  27. 05 Real Devices PROS • They are REAL Devices •

    There are things like Google Test Cloud, Test Droid and Amazon Device Farm CONS • You have to buy all the devices & hook them up • OR you have to pay a lot for cloud based use of devices which can be slow