Mobile Ops: Measuring Performance and Automating Workflow

Transcript

1. MOBILE Clay Smith, New Relic Mobile+Web Developer Conference OPS 4.27.2016
2. "OH MOBILE OPS? THE THING PEOPLE DON'T DO?" - Anonymous Software Engineer Good background from @patrickdubois: http://www.jedi.be/blog/2016/01/28/Mobile %20Delivery%20with%20a%20devops%20mindset/
3. "Very few [businesses] are well prepared to successfully deploy and maintain apps." -Accenture Mobile Survey https://adtmag.com/articles/2015/09/30/accenture-mobile-survey.aspx
4. HOW DO WE GET THERE?
5. MOBILE IS _________.
6. MOBILE IS _________. the future
7. MOBILE IS _________. vendor controlled
8. MOBILE IS _________. constrained
9. MOBILE IS _________. complicated
10. DEPLOYMENT LIFECYCLE TESTING BUILDING DESIGNING DEFINING PLANNING
11. DEPLOYMENT HARD PART TESTING BUILDING DESIGNING DEFINING PLANNING
12. #1 DEV PAIN? According to Spotify Internal Survey at Mobile Delivery Days 2016. BUILDS
13. SLOW GRADLE? ./gradlew assembleDebug --profile http://kevinpelgrims.com/blog/2015/06/11/speeding-up-your-gradle-builds/
14. LET'S TALK AUTOMATION Automate the painful, error-prone things.
15. CONTINOUS INTEGRATION Many, many options—Jenkins and beyond. https://gist.github.com/rmoriz/5296881
16. RELEASE TRAIN (GIT) https://www.natashatherobot.com/train-facebooks-model-for-mobile-project-management/ Master (in Play Store) Train/vNext Feature A
17. SOME CI ADVICE Automatically build and test on push Explore distributing beta builds Integrate with Slack/HipChat/etc Automatic Play Store release... maybe* http://simpleprogrammer.com/2009/12/30/continuous-integration-best-practices/ * https://wiki.jenkins-ci.org/display/JENKINS/Google+Play+Android+Publisher+Plugin https://github.com/Triple-T/gradle-play-publisher
18. DEVICE TESTING Cloud vs Self Hosted?
19. BUILDING IT YOURSELF? ‣ Consider http://square.github.io/spoon/ ‣ Run tests on multiple devices in parallel + HTML Summary ‣ Pros: Device Labs look super cool ‣ Cons: Device labs are hard to maintain
20. BECOME A FARMER? ‣ Many choices (search "mobile device testing") ‣ Ex: https://aws.amazon.com/device-farm/ ‣ Can run tests or fuzz ‣ Most have build system integration + APIs (./gradlew devicefarmUpload)
21. SPEED IS A Automating performance in Android FEATURE
22. CASE STUDY #1 A very large social network in Silicon Valley
23. WHAT THEY DID ‣ Desire to avoid performance regressions ‣ Belief in the ability to detect performance issues before release by using advanced techniques and automation ("the lab") ‣ Advanced distributed system using zookeeper and custom racks of devices https://code.facebook.com/posts/924676474230092/mobile-performance-tooling-infrastructure-at-facebook/
24. CASE STUDY #2 Doing it with New Relic + TravisCI + GitHub + AWS Device Farm https://github.com/smithclay/AndroidPerfTestExample
25. TRAVISCI GITHUB AWS DEVICE FARM NEW RELIC INSIGHTS ♥ SASS TravisCI runs on commits and uploads the latest build to AWS Device Farm, where performance analytics are collected from the app with New Relic. Build Test Collect
26. TRAVIS CI
27. AWS DEVICE FARM
28. HI, PERF DATA https://newrelic.com/insights
29. LAB ≠ REAL WORLD
30. MEASURE _________. real users
31. MEASURE _________. KPIs
32. SOME MOBILE KPIS ‣ Network requests (including failed) ‣ Startup time ‣ Blocked UI time ‣ Crashes, other errors ‣ Critical user interactions ‣ 3rd party services—what are they doing? https://medium.com/@mandrigin/ios-app-performance-instruments-beyond-48fe7b7cdf2
33. DEPLOYMENT COLLECT DATA HERE TESTING BUILDING DESIGNING DEFINING PLANNING
34. DEPLOYMENT AUTOMATE W/ CI HERE TESTING BUILDING DESIGNING DEFINING PLANNING
35. WHY? Users don't care how you release apps, but they want them to work and be fast.
36. THANK Slides posted shortly. @smithclay newrelic.com/mobile YOU 4.27.2016