I deploy on Fridays (and maybe you should too) (ConFoo 2024)

Danger, Will Robinson! Don’t deploy on Friday! Maybe your team's or company's policy?

Join me as I share from my experience. Using examples from my work on multiple real world projects, I’ll discuss resilience and operability, deployment strategies, pipelines and continuous testing. Discover the details of trunk based development, feature toggles, pair programming and other best practices and gain the confidence to deploy any day of the week!

Transcript

1. I DEPLOY ON FRIDAYS (AND MAYBE YOU SHOULD TOO) MICHIEL ROOK @MICHIELTCS
2. @michieltcs #StandWithUkraine DEPLOYING
3. @michieltcs #StandWithUkraine ON A FRIDAY
4. @michieltcs #StandWithUkraine DEPLOYING?
5. @michieltcs #StandWithUkraine ON A FRIDAY??
11. @michieltcs #StandWithUkraine
12. @michieltcs #StandWithUkraine BUT WHY?
13. @michieltcs #StandWithUkraine TRADITION
14. @michieltcs #StandWithUkraine FEAR
15. @michieltcs #StandWithUkraine RISK
16. @michieltcs #StandWithUkraine CONTROL
17. @michieltcs #StandWithUkraine CONFIDENCE & TRUST
18. @michieltcs #StandWithUkraine TIME TO REPAIR / RECOVER
19. @michieltcs #StandWithUkraine ON-CALL STRESS
20. @michieltcs #StandWithUkraine SO
21. @michieltcs #StandWithUkraine DON'T DEPLOY ON A FRIDAY!
22. @michieltcs #StandWithUkraine I USED TO SAY THAT TOO
23. @michieltcs #StandWithUkraine I LIKE MY WEEKENDS
24. @michieltcs #StandWithUkraine AND MY EVENINGS
25. @michieltcs #StandWithUkraine WHAT ABOUT BUGS?
26. @michieltcs #StandWithUkraine WHAT ABOUT 3 PM?
27. @michieltcs #StandWithUkraine WHAT ABOUT THURSDAY?
28. @michieltcs #StandWithUkraine FRIDAY IS 20% OF THE WEEK
29. @michieltcs #StandWithUkraine IMPROVE!
30. @michieltcs #StandWithUkraine IF IT HURTS DO IT MORE OFTEN
31. @michieltcs #StandWithUkraine FAILURE IS NOT THE OPPOSITE OF SUCCESS
32. SUCCESS CONSISTS OF GOING FROM FAILURE TO FAILURE WITHOUT LOSS OF ENTHUSIASM. Winston Churchill
33. @michieltcs #StandWithUkraine BIG STEPS
34. @michieltcs #StandWithUkraine FAIL BIG
35. @michieltcs #StandWithUkraine SMALL STEPS
36. @michieltcs #StandWithUkraine FAIL SMALL
37. @michieltcs #StandWithUkraine MAKE IT EFFORTLESS
38. @michieltcs #StandWithUkraine REDUCE RISK
39. @michieltcs #StandWithUkraine $ = REALIZED VALUE CREDITS TO @FGOULDING
40. @michieltcs #StandWithUkraine CONTINUOUS EVERYTHING
41. @michieltcs #StandWithUkraine
42. @michieltcs #StandWithUkraine DELIVERING VALUE TO USERS
43. @michieltcs #StandWithUkraine SAFELY & QUICKLY
44. @michieltcs #StandWithUkraine IN A SUSTAINABLE WAY
45. @michieltcs #StandWithUkraine EASY TO DEPLOY
46. @michieltcs #StandWithUkraine EASY TO TEST
47. @michieltcs #StandWithUkraine EASY TO INSPECT
48. @michieltcs #StandWithUkraine DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DELIVERY
49. @michieltcs #StandWithUkraine CODE IS ALWAYS IN A RELEASABLE STATE
50. @michieltcs #StandWithUkraine DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DEPLOYMENT
51. @michieltcs #StandWithUkraine IF YOU CAN DEPLOY ANY TIME
52. @michieltcs #StandWithUkraine THEN YOU CAN DEPLOY ON FRIDAY
53. @michieltcs #StandWithUkraine ELITE PERFORMERS Comparing the elite group against the low performers, we find that elite performers have… frequent code deployments 208 TIMES MORE time to recover from incidents 2,604 TIMES FASTER lead time from commit to deploy 106 TIMES FASTER change failure rate (changes are 1/7 as likely to fail) 7 TIMES LOWER Throughput Stability Source: 2019 State Of DevOps report
54. @michieltcs #StandWithUkraine BUT OUR CODE IS OLD
55. @michieltcs #StandWithUkraine BUT OUR CODE IS NOT UNDER TEST
56. @michieltcs #StandWithUkraine BUT OUR CODE IS IMPOSSIBLE TO CHANGE
57. @michieltcs #StandWithUkraine DEALING WITH "LEGACY"
58. STRANGLER PATTERN
59. @michieltcs #StandWithUkraine ORIGINAL MONOLITH DB @michieltcs
60. @michieltcs #StandWithUkraine ORIGINAL MONOLITH PROXY SERVICE ORIGINAL MONOLITH DB DB DB @michieltcs
61. @michieltcs #StandWithUkraine ORIGINAL MONOLITH PROXY SERVICE ORIGINAL MONOLITH ORIGINAL MONOLITH SERVICE SERVICE SERVICE PROXY DB DB DB DB DB DB @michieltcs
62. BRANCH BY ABSTRACTION
63. @michieltcs #StandWithUkraine CLIENT OLD MODULE CLIENT CLIENT @michieltcs
64. @michieltcs #StandWithUkraine CLIENT ABSTRACTION LAYER OLD MODULE CLIENT CLIENT @michieltcs
65. @michieltcs #StandWithUkraine CLIENT ABSTRACTION LAYER OLD MODULE CLIENT CLIENT @michieltcs
66. @michieltcs #StandWithUkraine CLIENT ABSTRACTION LAYER OLD MODULE CLIENT CLIENT ABSTRACTION LAYER NEW MODULE @michieltcs
67. @michieltcs #StandWithUkraine CLIENT ABSTRACTION LAYER OLD MODULE CLIENT CLIENT ABSTRACTION LAYER NEW MODULE @michieltcs
68. @michieltcs #StandWithUkraine CLIENT ABSTRACTION LAYER CLIENT CLIENT NEW MODULE @michieltcs
69. @michieltcs #StandWithUkraine CLIENT NEW MODULE CLIENT CLIENT @michieltcs
70. ON BRANCHES
71. @michieltcs #StandWithUkraine CONFLICTS
72. @michieltcs #StandWithUkraine "This pull request affects 31 files... could be a lot of conflicts if we don't merge it soon"
73. @michieltcs #StandWithUkraine
74. @michieltcs #StandWithUkraine CONTINUOUS INTEGRATION
75. @michieltcs #StandWithUkraine INTEGRATE TO MASTER AT LEAST DAILY
76. @michieltcs #StandWithUkraine DELAYING INTEGRATION
77. @michieltcs #StandWithUkraine TRUNK BASED DEVELOPMENT
78. @michieltcs #StandWithUkraine ONLY COMMIT TO MASTER
79. MERGING FEATURES?
80. @michieltcs #StandWithUkraine DECOUPLE DEPLOYMENTS FROM RELEASES
81. @michieltcs #StandWithUkraine FEATURE TOGGLES
82. @michieltcs #StandWithUkraine
83. @michieltcs #StandWithUkraine if (FeatureManager.isActive("my-feature")) {
 runSomeCodeForThisFeature();
 } else {
 runSomeOtherCodeNotForThisFeature();
 } @michieltcs
84. @michieltcs #StandWithUkraine PULL REQUESTS?
85. @michieltcs #StandWithUkraine CODE REVIEWS ARE USEFUL
86. @michieltcs #StandWithUkraine BUT: FLOW DELAY
87. PAIR PROGRAMMING
88. @michieltcs #StandWithUkraine CONTINUOUS INLINE CODE REVIEW
89. MOB PROGRAMMING
90. @michieltcs #StandWithUkraine PIPELINES
91. @michieltcs #StandWithUkraine AUTOMATION
92. @michieltcs #StandWithUkraine AUTOMATION AUTOMATION
93. @michieltcs #StandWithUkraine AUTOMATION AUTOMATION AUTOMATION
94. @michieltcs #StandWithUkraine COMPILE CHECKOUT TEST PACKAGE
95. @michieltcs #StandWithUkraine DEPLOY ACC VERIFY ACC DEPLOY PROD VERIFY PROD
96. @michieltcs #StandWithUkraine CONTINUOUS TESTING
97. @michieltcs #StandWithUkraine CONTINUOUS TESTING UNIT TESTS ACCEPTANCE TESTS E2E TESTS Cost Speed INTEGRATION TESTS
98. @michieltcs #StandWithUkraine CONTINUOUS TESTING UNIT TESTS ACCEPTANCE TESTS E2E TESTS Cost Speed Exploratory testing & user feedback Monitoring & alerting INTEGRATION TESTS
99. #StandWithUkraine @michieltcs "Testing shows the presence, not absence, of bugs." E D S G E R W. D I J K S T RA
100. @michieltcs #StandWithUkraine NO TEST SUITE DETECTS EVERY ISSUE
101. @michieltcs #StandWithUkraine PIPELINE SPEED = KEY
102. @michieltcs #StandWithUkraine DEPLOYMENTS
103. ROLLING UPDATE
104. @michieltcs #StandWithUkraine LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.0 SERVICE V1.0 @michieltcs
105. @michieltcs #StandWithUkraine LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.0 SERVICE V1.0 SERVICE V1.1 @michieltcs
106. @michieltcs #StandWithUkraine LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.1 SERVICE V1.0 SERVICE V1.0 @michieltcs
107. @michieltcs #StandWithUkraine LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.1 SERVICE V1.0 SERVICE V1.1 @michieltcs
108. @michieltcs #StandWithUkraine LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.1 SERVICE V1.1 SERVICE V1.0 @michieltcs
109. @michieltcs #StandWithUkraine LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.1 SERVICE V1.1 SERVICE V1.1 @michieltcs
110. @michieltcs #StandWithUkraine LOAD BALANCER ROLLING UPDATE SERVICE V1.1 SERVICE V1.1 SERVICE V1.1 SERVICE V1.0 @michieltcs
111. @michieltcs #StandWithUkraine LOAD BALANCER ROLLING UPDATE SERVICE V1.1 SERVICE V1.1 SERVICE V1.1 @michieltcs
112. @michieltcs #StandWithUkraine pipeline {
 stages {
 stage('Run tests') {
 sh "gradle check"
 }
 
 stage('Build docker image') {
 sh "docker build -t jobservice:${env.BUILD_NUMBER} ."
 sh "docker push jobservice:${env.BUILD_NUMBER}"
 }
 
 stage('Deploy staging') {
 sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER}
 -i staging deploy.yml"
 }
 
 stage('Deploy production') {
 sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER}
 -i prod deploy.yml"
 }
 }
 PIPELINE AS CODE image: registry.local/runner:latest
 
 stages:
 - test
 - images
 - deploy
 
 phpunit:
 stage: test
 script:
 - bin/phpunit
 
 images:
 stage: images
 script:
 - make images publish
 only:
 - master
 
 deploy:
 stage: deploy
 script:
 - make deploy
 only:
 - master
113. @michieltcs #StandWithUkraine
114. @michieltcs #StandWithUkraine FEEDBACK!
115. @michieltcs #StandWithUkraine DEPLOYING
116. @michieltcs #StandWithUkraine ON A FRIDAY
117. @michieltcs #StandWithUkraine YOU CAN DO IT TOO!
118. @michieltcs #StandWithUkraine THANK YOU FOR LISTENING! @michieltcs / michiel@michielrook.nl / @michielrook@mastodon.nl www.michielrook.nl