Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Observing and Understanding Failures - SRE Appr...

Observing and Understanding Failures - SRE Apprentices

Tammy Bryant Butow

May 12, 2021
Tweet

More Decks by Tammy Bryant Butow

Other Decks in Technology

Transcript

  1. Is it difficult to develop skills to observe and understand

    failures? Why is training from someone more experienced helpful? @tambryantbutow
  2. Luke after he met Yoda Lifts ship out of swamp

    after doing a handstand @tambryantbutow
  3. SRE Apprentice @tambryantbutow Padawan We created an SRE Apprentice Program

    to hire and train new SREs. Apprentices come from different backgrounds, for example we hired a Math Teacher who had completed a coding bootcamp (Hackbright Academy). We matched SRE Apprentices with SRE Teachers.
  4. @tambryantbutow When an Apprentice’s training is completed, they must pass

    the SRE interview-loop to become an SRE. They then continue to develop their skills and one day they will find a Padawan to train. SRE Teacher Padawan Jedi SRE Apprentice
  5. SRE Apprentice @tambryantbutow Padawan - Rona Chong The SRE apprenticeship

    was critical for my career - it was my foot into the door of the tech industry, when it can be hard to break in as a newcomer without the usual credentials. But getting your foot in the door is just the first step. To help set me up for success, Tammy was several things for me: a mentor, a source of emotional support, and an advocate. Find people who are truly dedicated to being there for others! I'm so glad that Tammy and her folks were there for me, checking in, thinking outside the box and pushing for growth and change in our communities - Rona Chong, Padawan // SRE Apprentice.
  6. SRE Apprentice @tambryantbutow SRE Teacher Padawan - Rona Chong Jedi

    - Tammy Bryant Butow Can feel disturbances in the force and use skills effectively to achieve desired results. Has a desire to learn everything she can about the force.
  7. SRE Teacher @tambryantbutow What makes someone suitable? • Set your

    Apprentice up for success • Be a mentor • Be a source of emotional support • Be an advocate • Check-in • Think outside the box • Push for growth and change in our communities • 2+ years SRE experience
  8. Psychological Safety @tambryantbutow What is the S.A.F.E.T.Y model and how

    can it help me teach my apprentice? Self Assessment: https://academy-bbl.com/safety-assessment/
  9. SRE Apprentice @tambryantbutow Padawan An approach to learning to observe

    and understand failures in Production Learn by training Learn by shadowing Learn by practice Learn by community * Mandalorian-style apprentice program featuring Grogu aka Baby Yoda and a Jedi trainer
  10. SRE Apprentice Learn by training: start with demo apps &

    lab environments before production @tambryantbutow
  11. Service Not Found Architecture @tammyxbryant Does blackholing a non-critical path

    service like the Recommendation Service cause unexpected failures for critical services like the Product Catalogue or Frontend?
  12. @tambryantbutow Does blackholing a non-critical path service like the Ad

    Service result in graceful degradation of the customer experience?
  13. Partner with your local community to teach the required skills

    engineers need @tambryantbutow holbertonschool.com
  14. Craft a daily email using python to send via cron

    job that includes key metrics; disk capacity, availability, latency etc. @tambryantbutow
  15. Build a web page that uses the PagerDuty API to

    display the most common alerts ordered by frequency - this enables us to use the Pareto Principle to improve system reliability. @tambryantbutow
  16. @tambryantbutow Tips for SRE Teachers: 1. Meet your apprentice where

    they are and help them get where they need to be. 2. Don't limit yourself as a mentor: Your apprentice's role isn't limited to code, so why should your mentorship? 3. Don't forget to establish open lanes of communication.
  17. @tambryantbutow . Tips for Apprentices: 1. One of the secrets

    to being effective is being willing to ask questions. 2. Don't just accept a task. 3. Unrealistic expectations happen all the time in software. 4. Invest in your relationships with your colleagues
  18. Thank you @tambryantbutow Get some chaos community stickers based on

    the path you’ll be taking: Padawan: gremlin.com/talk/padawan Jedi: gremlin.com/talk/jedi