Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DGW16 DevOps

ProdOps
June 20, 2016

DGW16 DevOps

Introduction to DevOps on DevGeekWeek'16

ProdOps

June 20, 2016
Tweet

More Decks by ProdOps

Other Decks in Technology

Transcript

  1. Leading in IT Education DevOps Why is everyone talking about

    it? Why is it so important all of a sudden?
  2. Leading in IT Education requests # of servers CPU aggregate

    techblog.netflix.com/2012/01/auto-scaling-in-amazon-cloud.html
  3. Leading in IT Education The community of developers whose work

    you see on the Web, who probably don't know what ADO or UML or JPA even stand for, deploy better systems at less cost in less time at lower risk than we see in the Enterprise. This is true even when you factor in the greater flexibility and velocity of startups. - Tim Bray, 2010 tbray.org/ongoing/When/201x/2010/01/02/Doing-It-Wrong
  4. Leading in IT Education “In the last week there were

    67 deploys of 496 changes by 18 people” – Flickr Dev Blog December 17th 2008 "10+ Deploys Per Day: Dev and Ops cooperation at Flickr" youtu.be/LdOe18KhtT4 at Velocity 2009
  5. Leading in IT Education “Between 12:00 AM and 11:59 PM

    on April 25, 2013, Quora released new versions of the site 46 times. This was a normal day for us.” - Quora engineering.quora.com/Continuous-Deployment-at-Quora “Deployment every 11.6s, 1,079 max in one hour. 10,000 mean number of hosts per deployment, with 30,000 maximum” - Amazon. com youtu.be/dxk8b9rSKOo “On the Google Consumer Surveys team, 8 minutes after you commit code it's live in production.” - Google developers.google.com/live/shows/772717729 “10+ deploys per day.” - John Allspaw, 2009 youtube.com/watch?v=LdOe18KhtT4
  6. Leading in IT Education "Some people get stuck on the

    word 'devops,' thinking it's just about development and operations working together. Systems thinking advises us to optimize the whole; therefore devops must apply to the whole organization, not only the part between development and operations." — Patrick Debois
  7. Leading in IT Education Continuous Delivery and Lean management practices

    directly correlate with high performing IT organisations.
  8. Leading in IT Education Does not matter if your apps

    are greenfield, brownfield or legacy -- as long as they are architected with testability and deployability in mind.
  9. Leading in IT Education 30x more deployments 200x faster lead

    times than their peers 60x the change success rate 168x faster to recover (MTTR) High performing vs. Low performing IT organisations - 2015
  10. Leading in IT Education Application Architecture Which of the following

    apply to the architecture of the primary application or service you are working on? from the 2015 state of devops report
  11. Leading in IT Education "Shipping first time code is like

    going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object- oriented or otherwise." — Ward Cunningham, 1992 c2.com/doc/oopsla92.html
  12. Leading in IT Education How can some companies perform so

    well that their industry counterparts are competitors in name only? Although they operate in the same industry, serve the same market, and even use the same suppliers, these extraordinary, high-velocity organizations consistently outperform all the competition ― and, more importantly, continually widen their leads. amzn.to/244hF7Z
  13. Leading in IT Education "While designing perfectly safe systems is

    likely beyond our abilities, safe systems are close to achievable when the four following conditions are met ... amzn.to/244hF7Z
  14. Leading in IT Education "While designing perfectly safe systems is

    likely beyond our abilities, safe systems are close to achievable when the four following conditions are met ... 1. See problems as they occur. amzn.to/244hF7Z
  15. Leading in IT Education "While designing perfectly safe systems is

    likely beyond our abilities, safe systems are close to achievable when the four following conditions are met ... 1. See problems as they occur. 2. Swarm and solve problems as they are seen to build new knowledge (fast). amzn.to/244hF7Z
  16. Leading in IT Education "If I suspect a problem, I

    pull the Andon cord and the whole production line stops immediately, and it doesn't start again until the issue has been resolved. I've pulled it thousands of times and it's good because it makes everyone personally responsible for producing the highest quality car." Bridie Tucker Team Member, Trim Assembly Toyota Burnaston, UK On a typical Toyota plant, the Andon cord is pulled ~3500 times every day.
  17. Leading in IT Education "While designing perfectly safe systems is

    likely beyond our abilities, safe systems are close to achievable when the four following conditions are met ... 1. See problems as they occur. 2. Swarm and solve problems as they are seen to build new knowledge (fast). 3. Spreading new knowledge throughout. amzn.to/244hF7Z
  18. Leading in IT Education "[The result of Admiral Rockover creating

    this capability] was that a young crew and their officers setting out for their first cruise on a US naval vessel benefits from the collective experience gained from over 5,700 reactor-years of experience behind them." — Dr. Steven Spear amzn.to/244hF7Z
  19. Leading in IT Education "While designing perfectly safe systems is

    likely beyond our abilities, safe systems are close to achievable when the four following conditions are met ... 1. See problems as they occur. 2. Swarm and solve problems as they are seen to build new knowledge (fast). 3. Spreading new knowledge throughout. 4. Leading by developing amzn.to/244hF7Z
  20. Leading in IT Education Don’t be a ‘Zombie’ organisation by

    Steven Spear huffingtonpost.com/steven-spear/dont-be-a-zombie-organiza_b_3306513.html youtu.be/P6kqnQZuIHg
  21. Leading in IT Education June 16, 2011. Velocity Conference ~15k

    deploys / day in May 2011 May 4, 2016. AWS Summit Stockholm ~50 million deploys / year ~136K deploys / day “Deployment every 11.6s, 1,079 max in one hour. 10,000 mean number of hosts per deployment, with 30,000 maximum” - Amazon. com youtu.be/dxk8b9rSKOo
  22. Leading in IT Education Design and Development ("the Product") Testing

    and Operations ("the Process") Create new products and services that improve the lives of customers by using hypothesis-driven development and fast feedback from users. Enable fast flow from Development to Production and reliable service to customers by standardizing work, reducing variability and batch sizes. Feature design and implementation may require work that has never been performed before. Estimates are highly uncertain. Outcomes are highly variable. Integration, Test and Deployment operations must be performed frequently (i.e. whenever an engineer checks in code into version control, multiple times per day) and as quickly as possible. Cycle times should be well-known and predictable. Outcomes should have low variability.
  23. Leading in IT Education Deployment “Lead Time” predicts... • ability

    of devs and ops to share a "common source of truth" • effectiveness of our automated testing in the deployment pipeline • ability to quickly deploy into production without causing chaos and disruption • ability to detect and correct problems through monitoring • ability for devs and ops to work together in a way that is win/win • how quickly devs can get feedback on their work
  24. Leading in IT Education What is the one question that

    predicts performance with startling accuracy?
  25. Leading in IT Education To what degree do we fear

    doing deployments? 1 we have no fear at all 7 we have existential fear
  26. Leading in IT Education lead time for changes release frequency

    time to restore service change fail rate How long is the delay between a request for a change, and a production system operating with that change implemented? How long does it take for an abnormal behavior in the system to be restored to the normal standard agreed way of operation? How many changes and features are being released to production in a fixed period of time? How often the system fails or service is disrupted?
  27. Leading in IT Education Topology of Org. Culture [Westrum 1994]

    Pathological Power-oriented Bureaucratic Rule-oriented Generative Performance-oriented Low cooperation Modest cooperation High cooperation Messengers shot Messengers neglected Messengers trained Responsibility shirked Narrow responsibilities Risks are shared Bridging discouraged Bridging tolerated Bridging encouraged Failure leads to scapegoating Failure leads to justice Failure leads to inquiry Novelty crushed Novelty leads to problems Novelty implemented
  28. Leading in IT Education characteristics of a generative culture devops

    practices high cooperation Cross-functional teams. Many organizations create cross-functional teams that include representatives from each functional area of the software delivery process (business analysts, developers, quality engineers, ops, security, etc.). This allows everyone to share the responsibility for building, deploying and maintaining a product. How to build a Generative Culture
  29. Leading in IT Education characteristics of a generative culture devops

    practices Messengers trained Blameless postmortems. By removing blame, you remove fear; and by removing fear, you enable teams to more effectively surface problems and solve them. Mistakes happen. Holding blameless postmortems is a valuable way to learn from mistakes. How to build a Generative Culture
  30. Leading in IT Education characteristics of a generative culture devops

    practices Risks are shared Shared responsibilities. Quality, availability, reliability and security are everyone’s job. One way to improve the quality of your services is to ensure that devs share responsibility for maintaining their code in production. The improvement in collaboration that comes from sharing responsibility inherently reduces risk: With more eyes on the software delivery process, it’s a given that some errors in process or planning will be avoided. Automation also reduces risk, and with the right tool choice, can enable collaboration. How to build a Generative Culture
  31. Leading in IT Education characteristics of a generative culture devops

    practices Bridging encouraged Breaking down silos. In addition to creating cross-functional teams, techniques for breaking down silos can include co-locating ops with the dev team; including ops in planning throughout the software delivery lifecycle; and implementing ChatOps.6 How to build a Generative Culture
  32. Leading in IT Education characteristics of a generative culture devops

    practices Failure leads to inquiry Blameless postmortems. Our response to failure shapes the culture of an organization. The more you focus on the conditions in which failures happen, as opposed to blaming individuals for failures, the closer you’ll get to creating a generative culture. How to build a Generative Culture
  33. Leading in IT Education characteristics of a generative culture devops

    practices Novelty implemented Experimentation time. Giving employees freedom to explore new ideas can lead to great outcomes. Some companies give engineers time each week for experimentation. Others host internal hack days or mini-conferences to share ideas and collaborate. This is how many new features and products have originated, and it shows how much value employees can generate for an organization when they’re released from habitual pathways and repetitive tasks. How to build a Generative Culture
  34. Leading in IT Education How Org. invest in Devops There

    are a number of ways IT leaders can invest in their teams: • Establish a dedicated training budget and make sure people know about it. Also, give your staff the latitude to choose training that interests them. • Encourage staff to attend technical conferences at least once a year and summarize what they learned for the entire team. • Set up internal hack days, where cross-functional teams can get together to work on a project. • Hold regular internal DevOps mini-conferences.
  35. Leading in IT Education How are you doing right now

    ... Q Do developers estimate task durations during sprint planning? A
  36. Leading in IT Education How are you doing right now

    ... Q What is the amount of planned vs. unplanned work during a sprint? A
  37. Leading in IT Education How are you doing right now

    ... Q How many value adding vs. non-value adding activities are done? A
  38. Leading in IT Education example of value adding activities: •

    deploy feature to production & get big new contract from a client. example of non-value adding activities: • bug fixes • rework • features never to be released • changes that make environments stop working
  39. Leading in IT Education Value Add vs. Non Value Add

    Value Adding Valuable Effort Costs Time Costs Money Adds Value VALUABLE Non-Value Adding Valueless Effort Costs Time Costs Money Adds No Value WASTE Obvious Waste where to draw the line between valuable and waste is not clear
  40. Leading in IT Education How are you doing right now

    ... Q Are there documented processes and standard procedures? A
  41. Leading in IT Education How are you doing right now

    ... Q How fast can a new employee get onboard and start working? A
  42. Leading in IT Education How are you doing right now

    ... Q Can developers turn features on and off in production? A
  43. Leading in IT Education “Make the bottlenecks work only on

    what will contribute to throughput today... not nine months from now.” – Eli Goldratt, The Goal
  44. Leading in IT Education “Break down barriers between departments. People

    in research, design, sales, and production must work as a team, to foresee problems of production and in use that may be encountered with the product or service.” – Dr. W. Edwards Deming "Out of the Crisis", 1982
  45. Leading in IT Education Types & Anti-Types of DevOps in

    organisations blog.matthewskelton.net/2013/10/22/what-team-structure-is-right-for-devops-to-flourish/ web.devopstopologies.com
  46. Leading in IT Education Reference Material Top Five DevOps learnings

    vimeo.com/165186982 Measure Efficiency, Effectiveness, and Culture to Optimize DevOps Transformation devopsenterprise. io/media/DOES_forum_metrics_102015.pdf DevOps Lessons Learned at Microsoft Engineering infoq.com/articles/devops-lessons-microsoft Devops: A Software Revolution in the Making? cutter.com/sites/default/files/itjournal/fulltext/2011/08/itj1108.pdf Ben Rockwood - SmartOS Operations www.youtube.com/watch?v=96PGoXHli3Q DevOps Solves Business Problems puppetlabs.com/blog/devops-solves-business-problems-gene-kims-top-aha-moments Projects are Evil and must be Destroyed evan.bottch.com/2010/08/29/projects-are-evil-and-must-be-destroyed/ DevOps is not a technology problem. DevOps is a business problem. dev2ops.org/2010/11/devops-is-not-a-technology- problem-devops-is-a-business-problem/
  47. Leading in IT Education Thank you! We invite you to

    join Operations Israel Facebook group on on.fb.me/Ops-IL we are hiring at [email protected] www.devops.co.il