Move Fast and Break Nothing

Transcript

1. break
   nothing
   move fast
   &
   

   View Slide

2. View Slide

3. Move fast and break things
   — Mark Zuckerberg, Facebook
   “ ”
   

   View Slide

4. Move fast and fix things
   — Mark Zuckerberg, Facebook
   “ ”
   

   View Slide

5. Move fast with stable infra
   — Mark Boringberg, Facebook
   “ ”
   

   View Slide

6. Move fast and be bold
   “
   — Miguel Velazquez, Facebook
   ”
   

   View Slide

7. Drop the ‘the’ — it’s cleaner.
   “
   — JUSTIN TIMBERLAKE SWOOOOOON
   ”
   

   View Slide

8. good philosophy
   moving quickly and safely is
   

   View Slide

9. what does it mean
   in practice?
   

   View Slide

10. @ holman
    

    View Slide

11. @ holman
    !
    

    View Slide

12. View Slide

13. View Slide

14. got me thinking about
    building environments
    where
    success
    is a byproduct
    

    View Slide

15. code
    3 areas
    , process& talk
    

    View Slide

16. code
    

    View Slide

17. move fast
    

    View Slide

18. people like fast
    better product
    

    View Slide

19. people like fast
    see what works
    

    View Slide

20. people like fast
    momentum
    

    View Slide

21. break things
    and
    

    View Slide

22. break things
    fix later and… stuff
    and
    

    View Slide

23. some things
    you can’t
    break
    

    View Slide

24. billing· permissions
    · upgrades· migrations
    

    View Slide

25. how do you
    move fast
    in dicey code?
    

    View Slide

26. we redid github’s
    permissions code
    

    View Slide

27. hella scary.
    

    View Slide

28. .01% errors
    unacceptable
    =
    

    View Slide

29. 0 errors
    mandatory
    =
    

    View Slide

30. how do we ship this
    fast + safe?
    

    View Slide

31. tests are great,
    of course
    

    View Slide

32. production can differ
    from what your tests expect
    

    View Slide

33. production can differ
    from what you even expect
    

    View Slide

34. ensure that your code
    doesn’t change production behavior
    

    View Slide

35. parallel
    codepaths
    

    View Slide

36. request
    

    View Slide

37. user.old_code
    always gets run
    request
    

    View Slide

38. user.new_code
    also gets run
    request
    user.old_code
    always gets run
    

    View Slide

39. user.new_code
    also gets run
    request
    user.old_code
    always gets run
    response
    

    View Slide

40. run both,
    compare
    

    View Slide

41. /github/dat-science
    p
    

    View Slide

42. science “new-auth” do | e |
    e.control { user.slow_auth }
    e.candidate { user.fast_auth }
    end
    

    View Slide

43. science “new-auth” do | e |
    e.control { user.slow_auth }
    e.candidate { user.fast_auth }
    end
    always runs and
    returns your result
    

    View Slide

44. can be run as
    a percentage
    science “new-auth” do | e |
    e.control { user.slow_auth }
    e.candidate { user.fast_auth }
    end
    

    View Slide

45. collects the
    two results
    science “new-auth” do | e |
    e.control { user.slow_auth }
    e.candidate { user.fast_auth }
    end
    

    View Slide

46. attempts vs. mismatches
    

    View Slide

47. 75th and 99th percentile performance
    

    View Slide

48. quickly iterate
    until you can
    prove safety
    

    View Slide

49. existing process
    build into your
    

    View Slide

50. what do you have today
    that you can already use?
    

    View Slide

51. each layer of
    process added
    is expensive
    

    View Slide

52. instead grow
    process laterally
    

    View Slide

53. tests rule everything
    around me
    

    View Slide

54. ci as code
    maintenance
    

    View Slide

55. failing test:
    removing html class
    without cleaning css
    

    View Slide

56. failing test:
    removing css without
    removing html class
    

    View Slide

57. failing test:
    adding that’s
    not on our cdn
    

    View Slide

58. failing test:
    invalid css (scss linting)
    

    View Slide

59. automates gruntwork
    out of code review
    

    View Slide

60. we test blog posts, too
    

    View Slide

61. failing test:
    images off our cdn
    

    View Slide

62. failing test:
    image non-retina
    or too big
    

    View Slide

63. failing test:
    using an oft-used
    phrase
    

    View Slide

64. pro-
    cess
    

    View Slide

65. what’s
    important?
    

    View Slide

66. feature launches need:
    · designers
    developers
    · product managers
    · qa
    · lawyers
    · marketing· ops
    

    View Slide

67. text files
    are rad
    

    View Slide

68. 
    

    View Slide

69. 
    anpp
    (apple new product process)
    

    View Slide

70. 
    (apple new product process)
    giant-ass
    checklist
    anpp
    

    View Slide

71. 
    (apple new product process)
    lists at tim-level
    and small team-level
    anpp
    

    View Slide

72. 
    (apple new product process)
    happens as
    very first step
    anpp
    

    View Slide

73. d
    

    View Slide

74. d
    ship lists
    

    View Slide

75. d
    ship lists
    happens often
    as last step
    

    View Slide

76. d
    ship lists
    all the todos for
    teams for their +1
    

    View Slide

77. View Slide

78. simple todo lists
    make you think less
    

    View Slide

79. ownership
    

    View Slide

80. don’t throw code
    over the wall
    

    View Slide

81. tie code to writers
    d
    

    View Slide

82. tie code to writers
    d
    notifies you
    if you screw up
    

    View Slide

83. View Slide

84. authors know
    their own code
    

    View Slide

85. authors should
    fix their bugs
    

    View Slide

86. how can
    your process
    help out?
    

    View Slide

87. tie code to writers
    

    View Slide

88. tie code to writers
    

    View Slide

89. tie code to writers
    chrome has
    owner files
    

    View Slide

90. tie code to writers
    dictates
    responsibility
    

    View Slide

91. tie code to writers
    makes
    mentors visible
    

    View Slide

92. tie code to writers
    d
    

    View Slide

93. tie code to writers
    d
    per-file
    ownership
    

    View Slide

94. tie code to writers
    d
    class BranchesController
    areas_of_responsibility :git
    end
    

    View Slide

95. View Slide

96. View Slide

97. tie code to writers
    d
    issues get assigned
    to the appropriate team
    

    View Slide

98. talk
    

    View Slide

99. more
    communication!
    yayyyyyyyycrap
    

    View Slide

100. email
     meetings
     chat
     issues
     im
     on-call
     video
     al
     

     View Slide

101. more is good
     better is better
     

     View Slide

102. most things
     aren’t emergencies
     

     View Slide

103. most things
     aren’t emergencies
     be mindful of time
     

     View Slide

104. measure impact
     of your actions
     be mindful of time
     

     View Slide

105. imagine:
     

     View Slide

106. 1. page a coworker for help
     2. they get woken up
     3. their phone takes a selfie of them
     4. selfie gets posted into chat
     

     View Slide

107. View Slide

108. would this make you respect
     your coworker’s time more?
     

     View Slide

109. add empathy to
     your process
     

     View Slide

110. empathy
     stems from being
     exposedto real pain
     

     View Slide

111. everyone should
     do some support
     

     View Slide

112. there’s a difference
     between
     reading about it
     helping with it
     &
     

     View Slide

113. institutional
     teaching
     

     View Slide

114. [we] have a responsibility to
     be teachers—that this should
     be a central part of [our] jobs
     “
     — Ed Catmull, Creativity, Inc.
     

     View Slide

115. …it’s just logic that someday
     we won’t be here.
     — Ed Catmull, Creativity, Inc.
     ”
     

     View Slide

116. more productive
     if workers are smart
     

     View Slide

117. how do you teach
     without making it lame?
     

     View Slide

118. chatops:
     learning by osmosis
     

     View Slide

119. app crashes. here’s a normal flow:
     

     View Slide

120. 1. employee gets paged
     2. they ssh into… something
     3. they fix it… somehow
     app crashes. here’s a normal flow:
     

     View Slide

121. no visibility means no teaching
     

     View Slide

122. no visibility means no improvement
     

     View Slide

123. d
     HUBOT
     

     View Slide

124. 1. employee gets paged
     2. they manage it in a chat room
     3. they fix it and people can watch
     here’s a better flow:
     app crashes.
     

     View Slide

125. visibility means teaching
     

     View Slide

126. visibility means improvement
     

     View Slide

127. real-life debugging
     

     View Slide

128. github’s wifi is down
     floor two, san francisco
     (right now!)
     

     View Slide

129. View Slide

130. View Slide

131. View Slide

132. View Slide

133. debug in the open
     

     View Slide

134. feedback
     

     View Slide

135. blue angels
     the
     feedback
     on
     

     View Slide

136. View Slide

137. two-way
     street
     

     View Slide

138. two-way
     street
     how well do
     you receive
     feedback?
     

     View Slide

139. two-way
     street
     how well do
     you give
     feedback?
     

     View Slide

140. improving
     feedbackacross an org
     

     View Slide

141. View Slide

142. caution
     with a degree of
     moving fast
     

     View Slide

143. need to be fast
     

     View Slide

144. need to be safe
     

     View Slide

145. takes effort
     to achieve both
     

     View Slide

146. @ holman
     

     View Slide