Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mobile release trains

Mobile release trains

- What is a release train
- What problems does it solve
- How to implement it for your mobile team

Luis G. Valle

March 04, 2021
Tweet

More Decks by Luis G. Valle

Other Decks in Programming

Transcript

  1. What is Travelperk? ⭐ Make business travelers happy ⭐ Help

    companies save money World’s largest inventory Flights, hotels, trains, accommodation All in one place, no more chasing receipts Search, book, pay, invoice, share and report — all on the same platform. Save money Cheaper options, low commissions, more data to optimise, Expensify integration
  2. Mobile Release trains at Travelperk Days 1-7 Development phase Day

    8 - 15.00 Code freeze QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint
  3. Work process for mobile engineering • One codebase per platform

    • Branches ◦ /release/x.x.x latest released code ◦ /master should always be releasable ◦ /feature/<feature-id> work branches • Small frequent PRs • Code reviews, pair programming • Feature flags
  4. Old release process Feature-based release ✅ Works great for one

    squad team ❌ Hold a release until desired feature is completed (no bug fixing) ❌ Multi-squad problems: ◦ Separate features block each other ◦ Delays and uncertainty ◦ Who owns QA for the single binary? is the feature done? Work on a feature Ship new app version Yes No
  5. “A release is the process of making a version of

    your app available to users” - Abraham Lincoln
  6. Stages of release processes Main branch where developers merge changes

    Merged changes are automatically tested There’s a CI server running those tests Continuous Integration Changes merged into the main branch are automatically deployed to a production-like environment Release to production is manual Continuous Delivery Changes merged into the main branch are automatically deployed to production No human intervention Continuous Deployment Source: https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment
  7. Stages of release processes (for mobile) Main branch where developers

    merge changes Merged changes are automatically tested There’s a CI server running those tests Continuous Integration Changes merged into the main branch are automatically deployed to a production-like environment Release to production is manual Continuous Delivery Changes merged into the main branch are automatically deployed to production No human intervention Continuous Deployment Source: https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment
  8. Why mobile releases are different from web/backend Distribution environment Updates

    are not instant: review process can take days and even reject the update Updates are not mandatory: users may never update to a new release Release as a single binary Apps are distributed as one single binary: we can’t update only parts of it To remove feature’s code you need to ship a new version of the whole project Rollback App stores do not allow rollbacks Any new bug means • New release for the whole app needs to be done (single binary) • It won’t be fixed today (we don’t control distribution environments)
  9. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Mobile release train
  10. Notion page with things to do before starting a release

    train Mobile release train: Checklist [ ] Assign release captains for Android and iOS [ ] Check which feature flags will need enabling [ ] Force upgrade after the release? [ ] Test cases for new features are added to Test Checklist [ ] Check all translations are in Phrase [ ] Release notes created and translated [ ] Changelog updated [ ] Release train announced in Slack [ ] Clone Amplitude release dashboard
  11. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    Train “departs” at 15.00 o’clock strictly Release branch cut from master New changes to master not allowed into the train from this point QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Mobile release train: Departure
  12. desc "Start a new Release train" lane :start_release do #

    Code freeze release_version = prompt(text: "Please provide a version number for the new release:") create_branch(from: "master", release_version: release_version) # Generate builds create_staging_build(branch_name: git_branch) create_production_build(branch_name: git_branch) UI.success "🍻 STAGING build and PRODUCTION build generated for current branch" end Mobile release train: Departure
  13. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    Release Captain drives this phase Manual* testing of production build. Core & feature tests shared in spreadsheet Feature flags checked here Only fixes accepted at this stage QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint * Working towards automation in Q1 2021 Mobile release train: QA
  14. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Allowed into the release branch if critical bugs are found during QA Non-critical bug fixes will board the next release train Needs agreement between Release Captain and PM Bug fix Mobile release train: bug fixing
  15. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    Semi-automated process Release notes handled by PM Validation process in stores can take up to 2-3 days QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Mobile release train: store submission
  16. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    1% > 2% > 5% > 20% > … > 100% to closely monitor adoption issues Staged rollout can be halted but never rolled back Hotfixes allowed for critical bugs -> 100% QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Hotfix Mobile release train: staged rollout
  17. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    Rollout reaches 100% Release tagged on Github Release branch merged back to master QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Mobile release train: Release
  18. desc "Finish release train: Tag version, create PR from release

    branch to master" lane :finish_release do # Get the release branch release_branch = sh("git branch -r | grep 'release' | tail -1 | tr -d ' ' | cut -d'/' -f2-").chomp # Check out release_branch and pull changes sh("git checkout #{release_branch}") # Get current version current_version = get_version_name # Tag release and push sh("git tag -a v#{current_version} -m '' && git push origin v#{current_version}") # Create Pull request open_pr_release(version: current_version) UI.success "🍻 Success!" end end Mobile release train: Finish a release
  19. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling
  20. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  21. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Triggered automatically from Bitrise at specific times ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  22. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Triggered automatically from Bitrise at specific times ➔ Start and finish release trains at fixed times ➔ Automatically create new release builds when hotfixes are merged into release branches and submit them ➔ Automatically run SAST/DAST tool nightly ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  23. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Triggered automatically from Bitrise at specific times ➔ Start and finish release trains at fixed times ➔ Automatically create new release builds when hotfixes are merged into release branches and submit them ➔ Automatically run SAST/DAST tool nightly ➔ Release dashboard is created automatically ➔ Publish release rollout status reports automatically on Slack ➔ Halt rollout if error rate goes over a threshold ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  24. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Triggered automatically from Bitrise at specific times ➔ Start and finish release trains at fixed times ➔ Automatically create new release builds when hotfixes are merged into release branches and submit them ➔ Automatically run SAST/DAST tool nightly ➔ Release dashboard is created automatically ➔ Publish release rollout status reports automatically on Slack ➔ Halt rollout if error rate goes over a threshold ➔ All train status changes are announced automatically ➔ Capacity to query: what’s the rollout status? Error rate? ➔ Capacity to action: halt release, generate stage build,... ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  25. 3 Execute the process manually first 4 Find automation opportunities

    2 Define a process that works for you 1 Identify what are the pain points for your company