Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How Does Yahoo! JAPAN Balance Development Speed...

How Does Yahoo! JAPAN Balance Development Speed and Service Quality?

Kanako Yasunaga (Yahoo! JAPAN / Technical Support Group, System Management Group, Technology Group / System Engineer)

https://tech-verse.me/ja/sessions/196
https://tech-verse.me/en/sessions/196
https://tech-verse.me/ko/sessions/196

Tech-Verse2022

November 18, 2022
Tweet

More Decks by Tech-Verse2022

Other Decks in Technology

Transcript

  1. Our Vision Software development in the recent years requires a

    speedy response to business changes. What is of importance is how quickly we can provide a new value to the customers without a drop in the quality of services. In this presentation, we will introduce examples of measurement and visualization of development speed and service quality, as well as improving the two in a well-balanced manner.
  2. Self - introduction 2004 Service development and operation Yahoo! Beauty

    / Yahoo! Ticket / Yahoo! BB Yahoo! Premium 2015 Promotion of platform development Company-wide Jenkins / Screwdriver / Splunk / Integrated console / In-house technical documentation / In-house community launch / In-house technical conference 2020 5FDIOPMPHZ4PMVUJPO%JWJTJPO Productivity visualization PF system development Analysis Kanako Yasunaga
  3. Agenda - Quantification of development speed and service quality -

    Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
  4. Measurement [1] DORA's research program, [2] Accelerate: The Science of

    Lean Software and DevOps: Building and Scaling High Performing Technology Organizations ※SSR is an abbreviation for site change success rate, a specification unique to Yahoo! JAPAN. four key metrics Development Speed Service Quality
  5. Measurement Change Failure Rate (SSR※) MTTR (Mean Time To Repair

    ) [1] DORA's research program, [2] Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations ※SSR is an abbreviation for site change success rate, a specification unique to Yahoo! JAPAN. Deploy Frequency Change Lead Time four key metrics Development Speed Service Quality
  6. How to measure coding build integrate test deploy operate improve

    Change Lead Time Deploy Frequency Change Failure Rate MTTR
  7. Change Lead Time low high early late *Survey results of

    371 products (421 systems) Conditions during (October- December 2020)
  8. Deploy Frequency low high low high *Survey results of 371

    products (421 systems) Conditions during (October- December 2020)
  9. MTTR ( Mean Time To Repair ) low high early

    late *Survey results of 371 products (421 systems) Conditions during (October- December 2020)
  10. Change Failure Rate SSR ※ SSR stands for Site change

    Success Rate low high low high *Survey results of 371 products (421 systems) Conditions during (October- December 2020)
  11. Cluster Deploy Frequency Change Lead Time MTTR Change Failure Rate

    (SSR ※) 1 month or more 1 month or more 1 week or more 45%〜 Once a week ~ once a month 1 week~ 1 month 1day〜 1week 30%〜45% At least once a week Less than a week Less than 1 day 0%〜30% Low Medium High ※ SSR stands for Site change Success Rate [3] Accelerate State of DevOps 2019
  12. Take a Balance Development Speed Service Quality High High Low

    Low Low Medium High *Survey results of 371 products (421 systems) Conditions during (October- December 2020)
  13. Agenda - Quantification of development speed and service quality -

    Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
  14. 下位ランク 中位ランク 上位ランク Best Practices Setting Content Master Base Development

    Is operated based on the best branch model (git flow, GitHub Flow and others) for the team. Code review is implemented for merging to branches (master branch and others) that will be the development base. Tests are implemented before merging to branches (master branch and others) that will be the development base. Branch survivability period is adequately short (generally within two weeks) Uses configuration management tools (Artifactory, Docker Registry, Chef and others) for packages, images, configuration management. Artifact Management Packages use CI/CD tools (Screwdriver.cd and others) Uses the same package, image and configuration management in all environments. Deployment Automation Deployment procedures are defined. Deployment procedures are the same for all environments. Deployment is automated. Deployment to production environment is implemented by triggering a merge to the branch to be the base for development. Deployment failure is automatically rolled back. Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Development Habits Survey クラス毎の統計データ 下位ランク 中位ランク 上位ランク [Survey question contents and cluster classification] *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Test Data Management Test data are all managed by version. Loose Coupling System coupling relationship is clear. Scope of effects caused by changes is clear. Incorporates initiatives for loose coupling. (Maintains backward compatibility, includes circuit breaker, and others) Adjustments caused by changes are under their own control. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
  15. Development Habits Survey クラス毎の統計データ [Survey question contents and cluster classification]

    *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes
  16. Development Habits Survey クラス毎の統計データ [Survey question contents and cluster classification]

    *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes
  17. Agenda - Quantification of development speed and service quality -

    Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
  18. Four Steps for Improvement Methods to improve development productivity 1.

    Know your own product numbers 2. Know your own product development habits 3. Comparison with high class 4. Improvements along with usual operations Photo: Afro
  19. 1.Know your own product numbers Change Failure Rate 0% Deploy

    Frequency 0.66回/月 MTTR 0% Change Lead Time 213.05 H Low Medium High High Development Speed Service Quality
  20. 2. Know your own product development habits x Master base

    development Artifact management Deployment automation Test Automation Test data management Loose coupling My product
  21. 3. Comparison with High class x Master base development Artifact

    management Deployment automation Test Automation Test data management Loose coupling High product My product
  22. 4. Improvements along with usual operations Uses the same package,

    image and configuration management in all environments.
  23. 4. Improvements along with usual operations Deployment procedures are the

    same for all environments. Deployment is automated.
  24. Quantitative Change Change Failure Rate 0% Deploy Frequency 0.66回/月 8回/月

    MTTR 0% Change Lead Time 213.05 H 134.4H Low Medium High High High High Development Speed Service Quality
  25. Before and After Improvement 改善前 改善後 Development Speed Service Quality

    High High Low Low High High Low Low Low High 6 months Service Quality Development Speed
  26. Agenda - Quantification of development speed and service quality -

    Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
  27. Knowledge Management Socialization Externalization Internalization Combination Tacit Tacit Explicit Explicit

    [4] The Knowledge Creating Company ( Knowledge Management (SECI Model) )
  28. Summary - Quantification of development speed and service quality -

    Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge