[R]Evolution of Open Source CI/CD Tools

How do Open Source and DevOps cultures impact Continuous Integration (and maybe not CD for once)? "Open source is eating the world". The same happens with the automation tools domain, including CI. There are many open source CI tools or open ecosystems on the market: Jenkins, Dagger, Argo projects, GitLab CI, Tekton, Dagger, GitHub Actions, etc., etc. There are many generations of these tools, with their architectures and use cases. We will talk about these tools and their evolution, from monolith services to cloud-native CI and Linux-way split to building blocks and interoperability.

What changed since CI was introduced 20 years ago? How do Open Source, InnerSource, and DevOps impact the landscape? What's next for the tools: cloud-native environments, machine learning, etc.? And how do open source foundations and other collaborations participate in the evolution of open source tools and facilitate it?

Transcript

1. [R]Evolution of open source CI/CD Tools by Oleg Nenashev, CDF/Jenkins Nov 08, 2021
2. > whoami @oleg_nenashev oleg-nenashev
3. 3 Agenda • Why do WE need CI/CD? • Generations of CI/CD tools • What’s now and what’s next? • What can you do after the talk? Ask me later: • How do we adjust as a company? • Jenkins roadmap These slides: @oleg_nenashev community.jenkins.io/t/new-t alk-evolution-of-open-source- ci-cd-tools/207 Discussion bit.ly/devoops2021-cicd
4. Disclaimer • Opinions are my own, happy to discuss • My talk does not represent opinions of my employer, the Continuous Delivery Foundation, or the Jenkins community • I present only open source projects today
5. What do Continuous Integration and Continuous Delivery mean? 5
6. CI or not CI? Code Pom.xml Tests git clone mvn clean verify $? != 0 : sendmail SCM CRONTAB Your Email Inbox
7. 7 CI or CD? ???
8. 8 CI or CD? Automation . Marketing CI CD …
9. 9 Disclaimer - CI/CD Tool ${YourCICDTool} Logo
10. Why do we need CI/CD? Selfish* view(s) on CI/CD * it’s fine!
11. ✔ Fast issue discovery? – Yes… ✔ Increased product quality? – Yes… ✔ Project transparency? – Yes… 11 Why do WE need CI/CD?
12. Managers • CI/CD works • Less resources spent by the team • Cool data and screenshots for slides
13. 13 http://imgs.xkcd.com/c omics/compiling.png WAITING FOR CI RESULTS BUILDING!
14. 14 We do NOT want • Merging changes • Reviewing broken code • Reviewing codestyle and code smells • Manually checking dependencies (licenses, security, etc.)
15. CI/CD Horrors 15 • Many notifications • Going through zillions of links • Digging into reports • Infrastructure failures • Flaky test • UI!
16. 16 We do NOT want • Complex UI and reports • Endless forms
17. 17 An engineer does not want to study Jenkins. He wants “build successful” What do developers want? Luigi is an automation engineer
18. 18 We do NOT want • Maintain CI/CD infrastructure • Troubleshoot infra failures • Think about the infra at all
19. ✔ Fast issue discovery? – Yes… ✔ Increased product quality? – Yes… ✔ Project transparency? – Yes… ✔ Contributor Experience (aka User Experience aka Developer Experience) 19 CI and CD 2021
20. • Continuous Integration and Continuous Delivery are about processes not tools • CI and CD are about you => Developer experience • Tools are here to help with implementing processes • Tools are about automation 20 Takeaways
21. 21 CI/CD Tools and their evolution
22. 22 Continuous Delivery Landscape (Short) https://landscape.cd.foundation/
23. 23 The reality... https://twitter.com/dastbe/status/ 1303858170155081728
24. 24 Types of CI/CD tools Frameworks “Semi-manufactures” • Many features provided out of the box • Can be extended • Maximum flexibility • Needs configuration “Ready to fly” • Everything available out of the box • Opinionated • Hard to customize
25. Generations of CI/CD Tools (Oleg’s IMHO) 25 III II I CRON . . .
26. Система Docs Plugins Configs Infra- structure Backend Samples Demos Infra- structure Infra- structure 26 ? without proper maintenance Generation III. Complexity, mostly frameworks
27. Generation III. Complexity Spaghetti Automation • A lot of customization • Duplication • Complexity • Lack of expertise • Difficult to maintain • Mistakes 27
28. 28 Generation III. New demands Shift left Generation III evolves … but not fast enough for early adopters
29. Generations of CI/CD Tools (Oleg’s IMHO) 29 IV III II I CRON ….
30. • Takes experiences of Generation III into account • Targets new environments • Scalability and public clouds in mind • Opinionated BUT: • Has to catch-up feature wise • Too opinionated for many 30 Generation IV IV
31. 31 Deployment, originally designed for Cloud-Native Bare-Metal (sic!) SaaS, Private and Public Clouds
32. [X] as code – part of CD and DevOps 32 [X] – Configuration, Documentation, Pipeline… Everything! Store [X] along with the project Versioning changes Testing [X] along with the project
33. Configuration as Code in CI/CD 33 Managing automation jobs (Pipeline as Code) Managing the system itself (infrastructure as code)
34. Pipelines 34
35. Pipelines 35 • Just another buzzword? • All stages as code • Various formats
36. 36 Pipeline may be different Pipeline Imperative Declarative Other…
37. 37 Declarative definitions are easier to use
38. Example - AppVeyor 38
39. 39 TeamCity KotlinDSL
40. 40 Imperative definitions are more universal and flexible
41. 41 Scripted Pipeline #!groovy def imageName = 'jenkinsciinfra/ircbot’ node('docker') { checkout scm // Немного магии для получения тэга Docker-образа sh 'git rev-parse HEAD > GIT_COMMIT' shortCommit = readFile('GIT_COMMIT').take(6) def imageTag = "build${shortCommit}» stage 'Build ircbot' withMavenEnv (["BUILD_NUMBER=${env.BUILD_NUMBER}:${shortCommit}"]) { sh 'make bot' // Make вызывает Maven } stage 'Build container' def whale = docker.build("${imageName}:${imageTag}”) stage 'Deploy container' whale.push() } https://github.com/jenkins-infra/ircbot
43. 43 Example - Jenkins Job DSL Jenkins Scripted Pipeline Jenkins Job Builder Groovy Groovy YAML … Jenkins Declarative Pipeline Groovy
44. 44 Gen III CI/CD tools evolved! … or became legacy
45. Evolution of generation III 45 IV III Evolution R.I.P. ???
46. != Jenkins Evolution Example 46
47. Modern Jenkins • Pipeline-as-Code • Configuration-as-Code • New plugins and integrations • Modern packaging • Distributions for public clouds 47
48. 48 CI/CD Tools Generation V V
49. CI/CD Tools. Generation V 49 • Learns from experiences of previous generations ◦ Opinionated but no longer general purpose • High specialization ◦ Unix way: “Do One Thing and Do It Well” • Mostly cloud native • Mostly open source as a tool or as ecosystem V
50. CI/CD Tools. Generation V - Examples 50 ??? Both IV and V
51. Example Keptn - Control-plane for DevOps automation of cloud-native applications 51 https://keptn.sh
52. Keptn // Sandbox project in the CNCF • Control plane, admin frontend/CLI • Observability, dashboards & alerting • SLO-driven multi-stage delivery • Operations & remediation 52 https://keptn.sh
53. Keptn • Sandbox project in the CNCF • Observability, dashboards & alerting • SLO-driven multi-stage delivery • Operations & remediation 53 https://keptn.sh
54. Oleg’s view 54 What’s now?
55. 55 https://cd.foundation/blog/202 1/02/01/trends-that-will-define- ci-cd-in-2021/
56. CI/CD Tools • Battle of generations, continued ◦ Generation 3 and 4 tools should keep evolving… or die ◦ Generation 5 keeps emerging 56
57. Getting out of Local Optimum 57 “Jenkins: Shifting gears”, Kohsuke Kawaguchi, 2018 jenkins.io/blog/2018/08/31/shifting-gears Generation IV Generation V
58. CI/CD Tools • Battle of generations, continued • Automation/Pipeline engines become a commodity 58
59. Pipeline engines become a commodity 59 Static Controllers Serverless build engines Jenkinsfile Runner Tekton (default) Classic Jenkins https://jenkins-x.io/ Jenkins X 3.x
60. CI/CD Tools • Battle of generations, continued • Automation/Pipeline engines become a commodity • Focus on emerging environments and use-cases ◦ Cloud native environments ◦ Container environments ◦ Kubernetes, Kubernetes, Kubernetes 60
61. Being Cloud Native • Best service for each need • Pay per use • “Infinite” scaling • Easy to use • Easy to maintain • Fast to develop 61
62. Example: Tekton https://tekton.dev/ Tekton is a powerful and flexible open-source framework for creating CI/CD systems, allowing developers to build, test, and deploy across cloud providers and on-premise systems. 62
63. Example: Tekton • Cloud Native • Kubernetes Native • Everything is in container • Steps and pluggability via containers https://tekton.dev/ 63
64. CI/CD Tools • Battle of generations, continued • Automation/Pipeline engines become a commodity • Focus on emerging environments and use-cases ◦ Cloud native environments ◦ Container environments ◦ Kubernetes, Kubernetes, Kubernetes • Focus on developer experience 64
65. Focus on developer experience 65
66. Jenkins X. Out of the Box experience • Build packs - Continuous Delivery • GitOps • Nexus, chartmuseum, monocular • Environments: Local, staging, production • IDE Integrations 66
67. CI/CD Tools • Battle of generations, continued • Automation/Pipeline engines become a commodity • Focus on emerging environments and use-cases ◦ Cloud native environments ◦ Container environments ◦ Kubernetes, Kubernetes, Kubernetes • Focus on developer experience • Software as a Service 67
68. Software as a Service (SaaS) • Ongoing switch to SaaS • SaaS does will no longer block you from using local resources ◦ GitHub Runner ◦ Gitlab Runner ◦ … • Internal SaaS Platforms ◦ Social coding and Inner Source ◦ Same principles as SaaS 68
69. Change is not just about tools. It is about culture 69
70. 70 Culture shift • Not just developers anymore • All roles are involved • Cross functional communications • TeamWork 💪
71. Culture Impact of DevOps • “Shift left” ◦ Developers manage pipelines ◦ Developers take more control • Focus on developer experience ◦ Continued evolution of Declarative syntax ◦ Imperative syntax for advanced users • Focus on interoperability • Being open source is an advantage 71
72. Culture Impact of Inner Source • Open Source culture gets adopted internally by the companies • Collaboration is key • Internal code collaboration platforms • CI/CD tools are critical to ensure developer experience and quality 72 https://innersourcecommons.org/
73. Open Source Won! 73
74. Going alone is no longer an option! Competing against communities, partnerships and foundations? Good luck! 74 Source: https://www.freshboxx.in/blog/technolo gy/free-and-open-source-software-foss -a-general-introduction
75. What’s next? Oleg’s view 75
76. THIS CITY NEEDS A HERO? 76
77. THIS CITY NEEDS A HERO? BETTER TOGETHER 77
78. 78 Continuous Delivery Foundation is here to help!
79. A neutral home for the next generation of continuous delivery collaboration
80. 80 https://cd.foundation/ CD Foundation is an open-source community improving the world's ability to deliver software with security and speed
81. CDF Mission • CDF believes in the power of Continuous Delivery to empower developers and teams and to produce high quality software more rapidly • CDF believes in the open-source solutions collectively addressing the whole Software Delivery LifeCycle • CDF fosters and sustains the ecosystem of open-source, vendor neutral projects through collaborations and interoperability • CDF advocates this idea and encourages collaborations among practitioners to share and improve their practices 81
82. 82 CDF Interoperability SIG • Focus on developer experience and their goals, not particular tools • How? ◦ Clarify what interoperability means for CI/CD ◦ Provide a neutral forum for projects ◦ Highlight and promote the needs of the users ◦ Explore synergies between, and enable collaboration across, the CI/CD projects ◦ Pursue solutions which are; loosely coupled, scalable, flexible, and technology agnostic ◦ Reduce the need to implement in-house solutions ◦ Attract/assist projects focused on interoperability github.com/cdfoundation/sig-interoperability
83. CDF Events SIG • Helping CI/CD systems to talk to each other in a standardized way • Goal: Provide interoperability through better communication and abstraction • How? ◦ Facilitate adoption of communication standards ◦ Promote adoption ◦ CDEvents - new standard for CD events over CloudEvents 83 https://github.com/cdfoundation/sig-events github.com/eiffel-community cloudevents.io
84. The End? ...almost 84
85. Takeaways • Automation “is eating the world” • Continuous Integration is an industry standard • Continuous Delivery is an industry standard • CI and CD are not just about tools • Tools are there, and they are evolving • The Continuous Delivery Foundation is here to help
86. Get ready to the era of open source tools • Become a good open source citizen! • Go beyond FOSS side projects, adapt your core business • Start an Open Source Program Office • Adopt InnerSource internally • Participate in foundations 86
87. What to do after the talk? 1. Take an open source CI/CD tool 2. Automate one of your projects 3. Try out another tool! 4. ...
88. It’s a great time to contribute ;) https://jenkins.io/participate
89. Contributing to the CDF • Join special interest groups! ◦ MLOps, Security,Events ◦ Interoperability, Best Practices • Spread the Word! • Join Ambassador program • Extend Landscape • Contribute to the CI/CD projects, not only member projects 89
90. Contributing to Jenkins jenkins.io/participate
91. And Keptn too! https://keptn.sh/community/contributing/ • Code • Advocacy • Adoption and Feedback • Integrations • CloudEvents Ecosystem 91
92. Contacts: E-mail: o.v.nenashev@gmail.com GitHub: oleg-nenashev Twitter: @oleg_nenashev QUESTIONS?
93. 93 Where is an enterprise ${COMPANY_NAME}? ${COMPANY_ NAME} logo
94. 94 Risks for ${COMPANY_NAME} • Perceived as classic enterprises focused company, risk of becoming legacy • Low adoption in the rapidly growing cloud native domain (aka “high performers”) • Increasing cost of traditional top-to-bottom sales • Growing competition from FOSS and Cloud Native vendors • Higher risk of churn by DevOps adopters and high performers (need to have control) ${COMPANY_ NAME} logo
95. Low adoption + Competition + Churn 95 * Not yet, but soon to become one for many companies
96. ${COMPANY_NAME}. Getting out of Local Optimum 96 “Jenkins: Shifting gears”, Kohsuke Kawaguchi, 2018 jenkins.io/blog/2018/08/31/shifting-gears Enterprises High performers
97. Going alone is no longer an option! Competing against communities, partnerships and foundations? Good luck! 97 Source: https://www.freshboxx.in/blog/technolo gy/free-and-open-source-software-foss -a-general-introduction
98. 98 Making ${COMPANY_NAME} a first class open source citizen Oleg’s Proposals
99. Key ideas • Become a good open source citizen! • Make ongoing open source participation visible • Embrace open source participation by your employees • Go beyond FOSS side projects, adapt your core business • Start an Open Source Program Office • Join and participate in foundations • Adopt InnerSource internally 99
100. Too many initiatives? Open Source Program Office! 100
101. 101 Source: https://mercan.mercari.com/ en/articles/25786/
102. OSPO - Projects and programs 102 • Define Open Source Usage Strategy • Define Open Source Investment Strategy - public or not • Open Source Stewardship • Foundation memberships representation, e.g. CDF, CNCF, Linux Foundation, OpenSSF, Apache Foundation, Eclipse Foundation • Facilitate community marketing and developer advocacy • Adopt open source culture and best practices for internal product development and collaboration. Be InnerSource!
103. Open Source Strategy • Public ◦ Communicate the vision to customers, end users and the community ◦ Improve the company’s image ◦ Communicate the vision to potential employees • Internal ◦ FOSS Usage guidelines and Licensing strategy ◦ Clear decision making and boundaries for FOSS work ◦ Minimum obstacles for open source contribution when reasonable ◦ ... 103
104. InnerSource at ${COMPANY_NAME}. Why? • Share best open source practices and knowledge with the teams • Improve cross-functional and cross-project communications • Remove barriers and break down silos via better collaboration • Improve overall employee experiences. Happy engineers do not quit (c) 104 innersourcecommons.org
105. InnerSource at ${COMPANY_NAME}. How? • Create public channels within the company for inner source and open source culture discussions • Start internal InnerSource Best Practices knowledge base • Start a regular internal newsletter with tips and links. Could reuse materials from InnerSource Commons and contribute back (see below) • Start regular internal meetups / office hours • Start a regular internal Hackergarten event for study and experiences sharing. Focus on internal products • …. 105 innersourcecommons.org
106. Social media is your friend! • Dedicated ${COMPANY_NAME} Community channels • Promote internal and external contributions • Collaboration with marketing, cross-posting and guerilla marketing • Life @ ${COMPANY_NAME} includes Open Source! • Same for the careers site! 106
107. More CDF and project SIGs • Security SIG • MLOps SIG • Best Practices SIG • Project SIGs and working groups ◦ jenkins.io/sigs ◦ jenkins.io/projects 107
108. CDF. Other Entities • Governing board ◦ Budgeting, staffing, etc. ◦ Infra sponsorships • Technical Oversight Committee ◦ Defining technical direction • Outreach Committee ◦ Marketing, events ◦ Ambassador program 108