The Road to Continuous Deployment: a case study (keynote at DevOpsPro Moscow 2017)

It’s a situation many of us are familiar with: a large legacy, monolithic application, limited or no tests, slow & manual release process, low velocity, no confidence… A lot of refactoring is required, but management keeps pushing for new features.

How to proceed? Using examples and lessons learned from a real-world case, I’ll show you how to replace a legacy application with a modern service-oriented architecture and build a continuous integration and deployment pipeline to deliver value from the first sprint. On the way, we’ll take a look at the process, automated testing, monitoring, master/trunk based development and various tips and best practices.

Transcript

1. THE ROAD TO CONTINUOUS DEPLOYMENT - A CASE STUDY MICHIEL ROOK @michieltcs
2. ▸ Developer, consultant, trainer, speaker ▸ @michieltcs
6. SAY MICROSERVICES ONE MORE TIME
12. WORKED FINE IN DEV OPS PROBLEM NOW
13. DEVOPS
14. DEVSECOPS
15. DEVSECQAOPS
16. DEVSECQABIZOPS
17. DEVSECQABIZNETOPS
18. DEVSECQABIZSYSNETOPS
19. DEVTESTSECQABIZSYSNETOPS
20. STORY TIME
22. @michieltcs
23. @michieltcs Large aging monolith
24. @michieltcs Large aging monolith Generates income
25. @michieltcs Large aging monolith Slow & complex Generates income
26. @michieltcs Large aging monolith Slow & complex Generates income Technical debt
27. @michieltcs
28. @michieltcs Manual releases
29. @michieltcs Manual releases Fragile tests
30. @michieltcs Manual releases Frequent issues Fragile tests
31. @michieltcs Manual releases Frequent issues Fragile tests Low velocity
32. GOALS @michieltcs
33. GOALS IMPROVE QUALITY 1 @michieltcs
34. GOALS IMPROVE QUALITY REDUCE TIME TO MARKET 1 2 @michieltcs
35. REFACTOR? REBUILD?
36. APPROACH ▸ API first ▸ Services per domain object (job, jobseeker, ...) ▸ Migrate individual pages @michieltcs
37. ORIGINAL MONOLITH DB @michieltcs
38. ORIGINAL MONOLITH PROXY SERVICE ORIGINAL MONOLITH DB DB DB @michieltcs
39. ORIGINAL MONOLITH PROXY SERVICE ORIGINAL MONOLITH ORIGINAL MONOLITH SERVICE SERVICE SERVICE PROXY DB DB DB DB DB DB @michieltcs
40. ARCHITECTURE FRONTENDS ARE SERVICES @michieltcs
41. ARCHITECTURE FRONTENDS ARE SERVICES SERVICES BEHIND LOAD BALANCERS @michieltcs
42. ARCHITECTURE FRONTENDS ARE SERVICES SERVICES BEHIND LOAD BALANCERS SERVICES IN CONTAINERS @michieltcs
43. IMPROVING
44. CONTINUOUS EVERYTHING
45. DEV BUILD / TEST CONTINUOUS INTEGRATION @michieltcs
46. DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DELIVERY @michieltcs
47. DEV BUILD / TEST STAGING / ACCEPTANCE PRODUCTION CONTINUOUS DEPLOYMENT @michieltcs
48. EVERY COMMIT GOES TO PRODUCTION
49. IF IT HURTS DO IT MORE OFTEN
50. SMALL STEPS @michieltcs
51. SMALL STEPS EARLY FEEDBACK @michieltcs
52. SMALL STEPS EARLY FEEDBACK REDUCE TIME TO RECOVER @michieltcs
53. SMALL STEPS EXPERIMENTS! EARLY FEEDBACK REDUCE TIME TO RECOVER @michieltcs
54. ONLY COMMIT TO MASTER
55. NO BRANCHES
56. NO BRANCHES REALLY.
59. DELAYING INTEGRATION
60. CONFLICTS
61. FEATURE TOGGLES
62. DECOUPLE DEPLOYMENTS
 FROM RELEASES
63. @michieltcs
64. FEATURE TOGGLE DEBT
65. CODE REVIEWS?
66. PAIR PROGRAMMING
67. CONTINUOUS
 INLINE CODE REVIEW
68. BUILD PIPELINE
69. CONNECT THE DOTS
70. AUTOMATE REPEATABLE THINGS
71. CONTINUOUS TESTING
72. CONFIDENCE
73. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS @michieltcs
74. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE UI TESTS @Test
 public void jobCannotBeFound() {
 when(jobRepository.getById(EXPECTED_JOB_ID))
 .thenReturn(null);
 
 JobService jobService = new JobService(jobRepository);
 
 assertNull(jobService.getById(EXPECTED_JOB_ID));
 verify(jobRepository).getById(EXPECTED_JOB_ID);
 } @michieltcs
75. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS @Test
 public void shouldFindJob() {
 expectedJob = loadFixture('active_job.yml');
 actualJob = repository.getById(expectedJob.getId());
 
 assertThat(actualJob, isA(Job.class));
 assertEquals(expectedJob.getId(), actualJob.getId());
 } @michieltcs
76. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS Scenario: Link to related job
 Given a job exists
 And there are related jobs available
 When that job is viewed
 Then a list of related jobs is shown
 And each related job links to the detail page of the related job @michieltcs
77. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS @michieltcs
78. UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI TESTS SMOKE
 TESTS Cost Speed Exploratory
 testing Monitoring @michieltcs
79. GOOD ENOUGH
80. PIPELINE SPEED = KEY
81. DEPLOYMENT
82. ROLLING UPDATE
83. LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.0 SERVICE V1.0 @michieltcs
84. LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.0 SERVICE V1.0 SERVICE V1.1 @michieltcs
85. LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.1 SERVICE V1.0 SERVICE V1.0 @michieltcs
86. LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.1 SERVICE V1.0 SERVICE V1.1 @michieltcs
87. LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.1 SERVICE V1.1 SERVICE V1.0 @michieltcs
88. LOAD BALANCER ROLLING UPDATE SERVICE V1.0 SERVICE V1.1 SERVICE V1.1 SERVICE V1.1 @michieltcs
89. LOAD BALANCER ROLLING UPDATE SERVICE V1.1 SERVICE V1.1 SERVICE V1.1 SERVICE V1.0 @michieltcs
90. LOAD BALANCER ROLLING UPDATE SERVICE V1.1 SERVICE V1.1 SERVICE V1.1 @michieltcs
91. PIPELINE AS CODE node {
 stage('Run tests') {
 sh "phpunit"
 sh "behat"
 }
 
 stage('Build docker image') {
 sh "docker build -t jobservice:${env.BUILD_NUMBER} ."
 sh "docker push jobservice:${env.BUILD_NUMBER}"
 }
 
 stage('Deploy staging') {
 sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER}
 -i staging deploy.yml"
 }
 
 stage('Deploy production') {
 sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER}
 -i prod deploy.yml"
 }
 } @michieltcs
92. @michieltcs
93. FEEDBACK!
94. RESULTS
95. @michieltcs Team acceptance
96. Feature toggle cap @michieltcs Team acceptance
97. Feature toggle cap Business alignment & agility @michieltcs Team acceptance
98. Feature toggle cap Business alignment & agility Focus on replacing legacy @michieltcs Team acceptance
99. Reduced number of issues 1 @michieltcs
100. Reduced number of issues 1 Build time
 < 10 min.
 (50+ per day) 2 4 @michieltcs
101. Reduced number of issues 1 Build time
 < 10 min.
 (50+ per day) Improved metrics & audience statistics 2 3 @michieltcs
102. Reduced number of issues 1 Build time
 < 10 min.
 (50+ per day) Improved metrics & audience statistics Increased confidence, velocity & fun 2 3 4 @michieltcs
103. THANK YOU! @michieltcs / mrook@fourscouts.nl
 www.fourscouts.nl
 www.michielrook.nl