Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[2020.11 Meetup] Lisa Crispin - Testing in DevOps

[2020.11 Meetup] Lisa Crispin - Testing in DevOps

DevOps Lisbon

November 24, 2020
Tweet

More Decks by DevOps Lisbon

Other Decks in Technology

Transcript

  1. @lisacrispin An Agile Thinking Tool for Building a DevOps Culture

    DevOps Lisbon, November 16, 2020 Lisa Crispin With material by Janet Gregory Copyright 2020
  2. @lisacrispin A little about me Part of OutSystems Quality Team

    [email protected] “Agile Testing for the Whole Team” now available as a remote course New!
  3. @lisacrispin My message today How using the agile testing quadrants

    model can identify risks and testing activities to help grow a quality-focused DevOps culture and succeed with continuous delivery / deployment. 3
  4. @lisacrispin What’s “DevOps”? • Whole team approach to quality –

    applied to Operations too • Small, frequent, low-risk changes • It’s a culture shift • No more throwing the release candidate over the wall to Ops • We build it – we run it – we own it!
  5. @lisacrispin What is continuous delivery? Continuous Delivery is the ability

    to get changes of all types— including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way. We achieve all this by ensuring our code is always in a deployable state, even in the face of teams of thousands of developers making changes on a daily basis. ~ Jez Humble and Dave Farley
  6. @lisacrispin Power of the quadrants • A thinking tool •

    Provides a common language • Emphasizes “whole team” • Plan testing at different levels • Tests for “Story Done” • Tests for “Feature Done” • Tests for “Release Done”
  7. @lisacrispin Functional examples (expected behavior / misbehaviors) Examples – 3rd

    party integrations Contract testing (on APIs) Unit testing Static analysis Component integration Test doubles for external integrations Data integrity Smoke tests on dev env. Exploratory testing Usability Monitoring Observability Testing in Production Recoverability Performance Load / Stress / Reliability Scalability, Safety Infrastructure as code
  8. @lisacrispin Try out the quadrants! Test a web app using

    Stripe to let donors make payments.
  9. @lisacrispin Apply the quadrants to a deployment pipeline • Infrastructure

    as code • Test environments • Team testing skills Photo by Sven Kucinic on UnSplash
  10. @lisacrispin Evaluate risks and plan mitigation strategies Release feature toggles

    – “hide” changes until confident to release • Enable incremental/iterative approach • For tests that critique the product What risks will you accept pending feedback in production? • scale, data, networking, configuration, user behavior
  11. @lisacrispin Apply quadrants to a simple deployment pipeline Local Unit

    tests Static analysis Test doubles -Conversations -Get examples -Create tests -Form a shared understanding of the problem Run unit tests Run api auto tests Observability Monitoring Test in Prod Run API auto tests Security, load, performance, recovery…. Run workflow auto tests UAT
  12. Create build failure alerts Script to auto build/deploy “preview” test

    envs Visualize & improve pipeline design Train everyone in exploratory testing Implement build status dashboard Define failure analysis runbook Define SLOs & SLIs Train everyone in failure analysis tools Script to send build results to Slack Evaluate auto test coverage Script to refresh test databases Evaluate SLI metrics in pre-prod
  13. @lisacrispin Going back to the quadrants… What do we need

    to do to move towards continuous delivery?
  14. @lisacrispin Improved communications No matter how it looks at first,

    it's always a people problem. -- Jerry Weinberg Most software problems are communication problems! The quadrants give us a framework to help with: • Common language • Shared understanding • Whole team approach to testing
  15. @lisacrispin Designers Getting every perspective • Programmers • SREs /

    platform engineers • Ops specialists • Business stakeholders • Testers • Designers • …. …….. SRE Ops Business Delivery Teams
  16. @lisacrispin Release with confidence 26 It’s all a balance, and

    it depends on your context Photo by Bekir Domex on Unsplash
  17. @lisacrispin Further reading https://agiletester.ca/more-agile-testing-the-book/ Download Chapter 8: Using Models to

    Help Plan Download Chapter 23: Testing in DevOps https://agiletester.ca/agile-testing-condensed-a-brief- introduction/ also has information on both of these topics
  18. Agile Testing Condensed, A Brief Introduction Agile Testing: A Practical

    Guide for Testers and Agile Teams More Agile Testing: Learning Journeys for the Whole Team By Janet Gregory and Lisa Crispin https://agiletester.ca https://agiletestingfellow.com Contact info https://lisacrispin.ca Email: [email protected] Twitter: lisacrispin 28