Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloud Friendly(?) Jenkins. How we failed to mak...

Cloud Friendly(?) Jenkins. How we failed to make Jenkins cloud native and what we learned?

Key takeaway: Being Cloud Friendly is a must, Cloud Native is optional. Likely YAGNI in your projects

Jenkins has been one of the most popular automation servers in the world. It is widely used for CI/CD purposes in cloud environments. You may have heard that Jenkins is not cloud native, and of course it is true. You may have heard that Jenkins controllers are a legacy monolith from the 2000s. That is not true, it is quite **cloud friendly** if used right. And yes we tried to make Jenkins cloud native, but it didn’t quite work. Why?

I have been involved in Jenkins architecture projects for more than ten years, including pluggable storage, multi-tenant Jenkins, cloud native agents, breaking down the controller monolith, Kubernetes integrations, etc., etc. Some of these efforts worked, some didn’t. But it was a great learning experience that can be used by developers and maintainers of other projects that consider re-platforming them to make cloud- and Kubernetes-native. I'll talk about the technical assumptions and mistakes we made in Jenkins, and how it worked. What to keep in mind when doing the re-platforming? And do you even need it?

Oleg Nenashev

March 18, 2024
Tweet

Resources

More Decks by Oleg Nenashev

Other Decks in Technology

Transcript

  1. > whoami @oleg_nenashev oleg-nenashev Dr. Nenashev / Mr. Jenkins Community

    builder Developer Tools Hacker #RussiansAgainstPutin #StandWithUkraine 2
  2. My Open [and not so] Communities Time * Once a

    contributor, [almost] always a contributor Neuchâtel 4
  3. We talk Jenkins! Disclaimer • My talk does not represent

    my employer’s opinion • It is not an official position of the Jenkins community • It is not related to the CloudBees announcements on HA Jenkins, but that’s a good stuff 5
  4. Disclaimer: Taking a break in Jenkins Away: personal events, ongoing

    war I participate in governance and GSoC (Co-)Maintainers welcome! I will be back! CDF Statement We stand with the people of Ukraine. Please assist humanitarian efforts for the Ukrainian people and those affected by the military invasion of Ukraine by supporting international aid organizations, including UNICEF. 6
  5. 7

  6. Flew past like plywood over Paris - idiom in the

    Russian language 10 J'ai survolé Paris comme du contreplaqué
  7. 11 • Having a baby • COVID • War in

    Ukraine • Solo Parenting Me missing Kubecon
  8. 15

  9. 16

  10. Generation III. New demands Shift left Generation III evolves …

    but not fast enough for early adopters 19
  11. Modern Jenkins • Pipeline-as-Code • Configuration-as-Code • New plugins and

    integrations • Modern packaging • Distributions for public clouds 22
  12. CI/CD Tools. Generation V • Learns from experiences of previous

    generations ◦ Opinionated but no longer general purpose • High specialization ◦ Unix way: “Do One Thing and Do It Well” • Mostly cloud native • Mostly open source as a tool or as ecosystem V 24
  13. Getting out of Local Optimum “Jenkins: Shifting gears”, Kohsuke Kawaguchi,

    2018 jenkins.io/blog/2018/08/31/shifting-gears Generation IV Generation V * Oleg’s CI tool generation definition 28
  14. Cloud Native Jenkins “General purpose CI/CD engine that runs on

    Kubernetes and embraces fundamentally different architecture & extensibility mechanisms” 29 https://flic.kr/p/75ncBF Kohsuke Kawaguchi, 2018
  15. Being Cloud Native • Best service for each need •

    Pay per use • “Infinite” scaling / Scale to zero • Easy to use • Easy to maintain • Fast to develop 31
  16. Controller as a SPoF Macroservice 33 Permanent agents On-demand agents

    from cloud providers Web UI Jenkins Controller
  17. Cloud Native Special Interest Group • Founded in July 2018

    • jenkins.io/sigs/cloud-native • R.I.P in 2022 Goal: Improve Jenkins to run on Cloud environments as a "Cloud Native" application 34 ?
  18. Cloud Native SIG Projects 35 • Pluggable Storage • Jenkins

    X 1.x • Configuration-as-Code • OTel and Cloud Events Integrations • Multi-tenant Jenkins Controller • Operator for Kubernetes • Jenkinsfile Runner (portable Pipeline)
  19. 36

  20. Jenkins X. Out of the Box experience • Build packs

    - Continuous Delivery • GitOps • Nexus, chartmuseum, monocular • Environments: Local, staging, production • IDE Integrations 37
  21. Pipeline engines become a commodity 38 Static Controllers Serverless build

    engines Jenkinsfile Runner Tekton (default) Classic Jenkins https://jenkins-x.io/ Jenkins X 3.x
  22. Cloud Native Jenkins “General purpose CI/CD engine that runs on

    Kubernetes and embraces fundamentally different architecture & extensibility mechanisms” 40 https://flic.kr/p/75ncBF Kohsuke Kawaguchi, 2018
  23. Jenkinsfile Runner • Binary and base Docker image • Runs

    a Pipeline job and shuts down • No Web UI* • On-demand packaging as a container or JAR github.com/jenkinsci/jenkinsfile-runner 42
  24. 44

  25. • Not a CI/CD solution on its own Limitations Triggers

    Webhook receivers Log services Result Storage Web UI 45
  26. • Not a CI/CD solution on its own • No

    Web UI • No Queue • No build throttling, cross-controller communication, etc. • Designed for a single-container use Limitations 46
  27. Pluggable Storage: current state • Artifacts • Build logs •

    Configuration • Credentials • Builds • Jobs • Test results • Code coverage • Static analysis • System logs • Task logs • Fingerprints • ... Legend done & available partial not started jenkins.io/doc/book/using/pluggable-storage/ 48
  28. 49

  29. Jenkins Operator for Kubernetes 52 • Actually, two operators from

    different vendors • Core Jenkins vendors were busy with other projects • We tried to set up community governance too late
  30. 53

  31. 54

  32. 59 • Infra and CI maintenance are overhead • SaaS

    or service • Simple for simple cases • Affordable for complex
  33. Система Docs Plugins Configs Infra- structure Backend Samples Demos Infra-

    structure Infra- structure 60 ? without proper maintenance What they don’t want
  34. Being Cloud Native • Best service for each need •

    Pay per use • “Infinite” scaling / Scale to zero • Easy to use • Easy to maintain • Fast to develop 62
  35. Being Cloud Friendly • Best service for each need Best

    service for your needs • Pay per use • “Infinite” scaling / Scale to zero Scales as needed • Easy to use in the cloud • Easy to maintain in the cloud • Fast to develop in the cloud 63
  36. There are many who can do cloud native, but only

    a few who can do shared context services 65 Viktor Farcic devopsparadox.com
  37. 69

  38. OTel in Jenkins • Traces can go through different tools,

    not just a single one • OTel Tracing is a DAG Source: plugins.jenkins.io/opentelemetry/ 71
  39. Tekton Client Plugin Pipeline Engine is no a commodity Tekton

    provides the ability to create “k8s-style resources for declaring CI/CD-style pipelines”. It is the go to CI/CD system for creating Pipelines on Kubernetes. You can create Tekton Resources from Jenkins Recent v1 Release ! Blogpost : Easily reuse Tekton and Jenkins X from Jenkins 73
  40. • Multiple checks • Status reporting • Summary and Details

    • GitHub Markdown support • Annotations https://github.com/jenkinsci/gradle- plugin/pull/382/checks?check_run_i d=19876936412 76
  41. The future is ahead! “Jenkins: Shifting gears”, Kohsuke Kawaguchi, 2018

    jenkins.io/blog/2018/08/31/shifting-gears Generation IV Generation V * Oleg’s CI tool generation definition 77
  42. Takeaways • One doesn’t have to be Cloud Native. Think

    Cloud Friendly first • Scalability is not a goal on its own, scale what needs to scale • Focus on user and developer experience • Pay your architecture debt 78