Andreas Caternberg on Jenkins Pipelines

Transcript

1. Let’s Build a Jenkins Pipeline Andreas Caternberg
2. Download Slides Now https://gist.github.com/cccaternberg/3984e4a3d1a84ec6faffbd728ce6c247
3. About Your Guide Andreas Caternberg Senior Consultant at CloudBees @acaternberg
4. Plan for today
5. CloudBees Jenkins Enterprise © 2017 CloudBees, Inc. All Rights Reserved • Community Innovation • Jenkins for the Enterprise
6. What we are going to cover • What is a pipeline • Compare scripted pipeline to declarative pipeline • Create a declarative pipeline • Learn about Blue Ocean • Share pipeline resources • Extra scripted pipeline information © 2017 CloudBees, Inc. All Rights Reserved
7. What is a Jenkins pipeline • Formerly known as Jenkins Workflow, Jenkins Pipeline was introduced in 2014 • New job type – a single Groovy script using an embedded DSL - no need to jump between multiple job configuration screens to see what is going on • Durable - keeps running even if Jenkins master is not • Distributable – a Pipeline job may be run across an unlimited number of nodes, including in parallel • Pausable – allows waiting for input from users before continuing to the next step • Visualized - Pipeline Stage View provides status at-a-glance dashboards to include trending © 2017 CloudBees, Inc. All Rights Reserved
8. What is a declarative pipeline? • A syntax for Jenkins Pipelines designed to… –Have smarter default behavior for most use cases –Make Pipelines more structured –Provide better error reporting and handling © 2017 CloudBees, Inc. All Rights Reserved
9. Have a smarter behavior • Default SCM checkout • Built with Docker and containers in mind from day one • No more need for try/catch for notifications, etc. • More intuitive ways to specify job properties, triggers and parameters • Streamlined syntax for conditional execution of stages © 2017 CloudBees, Inc. All Rights Reserved
10. More structure in Pipelines • Separate configuration from actual steps to be executed • Everything happens in a stage • Easier to look at any Declarative Pipeline and understand what it’s doing - more consistency! • Enables round-tripping with the visual editor (more on this later!) • Built with and for Blue Ocean visualization © 2017 CloudBees, Inc. All Rights Reserved
11. Better error reporting • Syntax and validation errors reported at the very beginning of the build • No more waiting until a build has gone an hour to discover that you misspelled “timeout”! • Errors report as compilation errors, pointing to the specific code or text that’s causing problems • Eliminates many potentially confusing stack traces (though not all!) • Validates things like syntax, required configuration, step parameter types, and more © 2017 CloudBees, Inc. All Rights Reserved
12. What happens to Scripted Pipeline? • In short? Nothing. • We’re now calling “traditional” Pipeline “Scripted Pipeline” to distinguish it from Declarative Pipeline • All one execution engine behind the scenes ◦Declarative is built on top of Scripted Pipeline, not replacing it ◦Blue Ocean and Stage View don’t distinguish between a Scripted Pipeline or a Declarative Pipeline • Scripted Pipeline still used *inside* Declarative Pipeline in steps blocks, post blocks ◦You can copy the contents of a Declarative steps block into a Scripted Pipeline (inside a node block, of course!) and it’ll Just Work. © 2017 CloudBees, Inc. All Rights Reserved
13. Benefits of Declarative for new users • Lower barrier of entry than Scripted Pipelines • Human readable • Does not require Groovy-specific expertise • Create and edit Declarative Pipelines via the upcoming UI Editor © 2017 CloudBees, Inc. All Rights Reserved
14. Benefits of Declarative for existing users • Uses same engine as Scripted, so existing investments still pay off • Expand usage and usability ◦Less burden on “experts” ◦Easier empowerment of other users • Easier collaboration and code review • Lintable! • Separation of Jenkins infrastructure-related configuration from steps to execute © 2017 CloudBees, Inc. All Rights Reserved
15. Syntax Overview © 2017 CloudBees, Inc. All Rights Reserved
16. Pipeline Comparison © 2017 CloudBees, Inc. All Rights Reserved Scripted Pipeline node { try { stage('Build') { checkout scm docker.image('ubuntu').inside { sh 'mvn clean install' } } } catch(exc) { mail to:'me@ex.com', subject:'FAILURE:' } finally { deleteDir() } } Declarative Pipeline pipeline { agent { docker 'ubuntu' } stages { stage('Build') { steps { sh 'mvn clean install' } } } post { always { deleteDir() } failure { mail to:'me@ex.com', subject:'FAILURE:' } } }
17. First example © 2017 CloudBees, Inc. All Rights Reserved https://gist.github.com/cccaternberg/1d2169d445d04d5d6264a241fbc53e8e
18. © 2017 CloudBees, Inc. All Rights Reserved
19. © 2017 CloudBees, Inc. All Rights Reserved
20. © 2017 CloudBees, Inc. All Rights Reserved
21. © 2017 CloudBees, Inc. All Rights Reserved
22. © 2017 CloudBees, Inc. All Rights Reserved
23. © 2017 CloudBees, Inc. All Rights Reserved
24. Second Example © 2017 CloudBees, Inc. All Rights Reserved https://gist.github.com/cccaternberg/b45e58163725c3ca81f76a55e1328009
25. © 2017 CloudBees, Inc. All Rights Reserved
26. © 2017 CloudBees, Inc. All Rights Reserved
27. © 2017 CloudBees, Inc. All Rights Reserved
28. © 2017 CloudBees, Inc. All Rights Reserved
29. Time to build a pipeline © 2017 CloudBees, Inc. All Rights Reserved
30. Get Ready • Log into our CloudBees Jenkins Operations Center http://dpa.beedemo.net/pipeline- workshop/ • Create an account © 2017 CloudBees, Inc. All Rights Reserved
31. Get Ready • Log in • Go to the LBAJP server © 2017 CloudBees, Inc. All Rights Reserved
32. Create a Pipeline Job © 2017 CloudBees, Inc. All Rights Reserved • Select ”New Item” • Name your job
33. A basic pipeline pipeline { agent any stages { stage('Build') { steps { echo 'Hello from Build' } } stage('Test') { steps { echo 'Testing Testing 123' } } stage('Deploy') { steps { echo 'Deploy some things' } } } } © 2017 CloudBees, Inc. All Rights Reserved
34. Add an agent © 2017 CloudBees, Inc. All Rights Reserved pipeline { agent any stages { pipeline { agent { label ‘docker’ } stages {
35. Stash Files © 2017 CloudBees, Inc. All Rights Reserved stages { stage('Build') { steps { echo 'Hello from Build’ writeFile(file: 'config', text: 'version=1', encoding: 'UTF-8') stash(name: 'pom-config', includes: 'config') } } stage('Test') { environment { configValue = readFile(encoding: 'UTF-8', file: 'config') } steps { echo 'Testing Testing 123’ unstash 'pom-config' sh 'echo "$configValue"' } }
36. Create an input step stage('Deploy') { steps { echo 'Deploy some things' input 'Do you want to deploy?' echo 'deployed' } } © 2017 CloudBees, Inc. All Rights Reserved
37. Blue Ocean © 2017 CloudBees, Inc. All Rights Reserved
38. Blue Ocean Visualization • Some special smarts for optimized visualization • Blue Ocean keeps built-in stages for things like checkout, Docker image prep, post-build actions out of your way unless there’s a problem • Special marking of stages that have been skipped due to either an unsatisfied when condition or an earlier failed stage © 2017 CloudBees, Inc. All Rights Reserved
39. Blue Ocean Editor © 2017 CloudBees, Inc. All Rights Reserved
40. © 2017 CloudBees, Inc. All Rights Reserved
41. © 2017 CloudBees, Inc. All Rights Reserved
42. © 2017 CloudBees, Inc. All Rights Reserved
43. Reference Card https://www.cloudbees.com/sites/default/files/declarative-pipeline-refcard.pdf © 2017 CloudBees, Inc. All Rights Reserved
44. Resources on Declarative • Documentation https://jenkins.io/doc/ • Examples https://github.com/jenkinsci/pipeline-examples • Plugin source https://github.com/jenkinsci/pipeline-model-definition-plugin © 2017 CloudBees, Inc. All Rights Reserved
45. Scripted Pipeline © 2017 CloudBees, Inc. All Rights Reserved
46. Pipeline: a new job type -<YourName>
47. Key Benefits ✓ Long-running ✓ Durable ✓ Scriptable ✓ One-place for Everything Pipeline: a new job type ✓ Makes building Pipelines Simple
48. Domain Specific Language Simple, Groovy-based DSL (“Domain Specific Language”) to manage build step orchestration Can be defined as DSL in Jenkins or as Jenkinsfile in SCM Groovy is widely used in Jenkins ecosystem. You can leverage the large Jenkins Groovy experience If you don’t like/care about Groovy, just consider the DSL as a dedicated simple syntax
49. Snippet Generator 49
50. Create a Pipeline - core steps 50 stage - group your steps into its component parts and control concurrency node - schedules the steps to run by adding them to Jenkins' build queue and creates a workspace specific to this pipeline sh (or bat) - execute shell (*nix) or batch scripts (Windows) just like freestyle jobs, calling any tools in the agent
51. Create a Pipeline stage('build') { echo 'hello from jenkins master' node { sh 'echo hello from jenkins agent' } } stage('test') { echo 'test some things' } stage('deploy') { echo 'deploy some things' } 51
52. Lab Exercise: Checkout from SCM
53. Files stash - store some files for later use unstash - retrieve previously stashed files (even across nodes) writeFile - write a file to the workspace readFile - read a file from the workspace 53
54. Checkout from SCM and stash Files stage('build') { node { git 'https://github.com/beedemo/mobile-deposit-api.git' writeFile encoding: 'UTF-8', file: 'config', text: 'version=1' stash includes: 'pom.xml,config', name: 'pom-config' } } stage('test'){ node { unstash 'pom-config' configValue = readFile encoding: 'UTF-8', file: 'config' echo configValue } } 54
55. try up to N times retry(5) { // some block } wait for a condition waitUntil { // some block } Flow Control wait for a set time sleep time: 1000, unit:'NANOSECONDS' timeout timeout(time: 30, unit: 'SECONDS'){ // some block }
56. You can also rely on Groovy control flow syntax! while(something) { // do something if (something_else) { // do something else } } Flow Control 56 try{ //some things }catch(e){ // }
57. Advanced Flow Control input - pause for manual or automated approval parallel - allows simultaneous execution of build steps on the current node or across nodes, thus increasing build speed parallel 'quality scan': { node {sh 'mvn sonar:sonar'} }, 'integration test': { node {sh 'mvn verify'} } checkpoint - capture the workspace state so it can be reused as a starting point for subsequent runs
58. Lab Exercise: Input and Checkpoints
59. Input Approval & Checkpoints 59 stage('deploy') { input message: 'Do you want to deploy?' node { echo 'deployed' } }
60. Input Approval & Checkpoints 60 checkpoint 'testing-complete' stage('approve') { mail body: "Approval needed for '${env.JOB_NAME}' at '${env.BUILD_URL}'", subject: "${env.JOB_NAME} Approval", to: "ops@yourcompanyhere.com" timeout(time: 7, unit: 'DAYS') { input message: 'Do you want to deploy?', parameters: [string(defaultValue: '', description: 'Provide any comments regarding decision.', name: 'Comments')], submitter: 'ops' } }
61. Lab Exercise: Tool Management
62. tool Step •Binds a tool installation to a variable •The tool home directory is returned •Only tools already configured are available def mvnHome = tool 'M3' sh "${mvnHome}/bin/mvn -B verify" 62
63. Use Docker Containers •The CloudBees Docker Pipeline plugin allows running steps inside a container •You can even build the container as part of the same Pipeline docker.image('maven:3.3.3-jdk-8').inside() { sh 'mvn -B verify' } 63
64. Lab Exercise: Pipeline as Code
65. Pipeline script in SCM 65
66. Pipeline Multibranch 66
67. Pipeline Shared Libraries
68. What Next?
69. More Advanced Steps Send email mail body: 'Uh oh.', subject: 'Build Failed!', to: 'dev@cloudbees.com' step([$class: 'Mailer', notifyEveryUnstableBuild: true, recipients: 'info@cloudbees.com']) Deploy to Amazon Elastic Beanstalk wrap([$class: 'AmazonAwsCliBuildWrapper', credentialsId: 'aws-beanstalk- credentials', defaultRegion: 'us-east-1']) { sh 'aws elasticbeanstalk create-application-version' } Integrate with Jira jiraComment(issueKey: "EX-111", body: "Job '${env.JOB_NAME}' ($ {env.BUILD_NUMBER}) built. Please go to ${env.BUILD_URL}.")
71. Gartner: “Using Docker to Run Build Nodes is Ideal.” Agent Master z Agent Agent
72. Accelerating CD with Containers Workflow CD Pipeline Triggers: ✓ New application code (feature, bug fix, etc.) ✓ Updated certified stack (security fix in Linux, etc.) ✓ Will lead to a new gold image being built and available for … TESTING … STAGING … PRODUCTION ✓ All taking place in a standardized/similar/consistent OS environment + Jenkins Pipeline TEST STAG E PRODUCTIO N App <code> (git, etc.) Gold Docker Image (~per app) <OS config> Certified Docker Images (Ubuntu, etc.) <OS config>
73. Jenkins Pipeline Docker Commands • withRegistry Specifies which Docker Registry to use for pushing/pulling • withServer Specifies which Server to issue Docker commands against • Build Builds container from Dockerfile • Image.run Runs a container • Image.inside Runs container and allows you to execute command inside the container • Image.push Pushed image to Docker Registry with specified credentials
74. Pipeline - Docker Example stage 'Build' node('docker-cloud') { checkout scm docker.image('java:jdk-8').inside('-v /data:/data') { sh "mvn clean package" }} stage 'Quality Analysis' node('docker-cloud') { unstash 'pom' parallel( JRE8Test: { docker.image('java:jre-8').inside('-v /data:/data') { sh 'mvn -Dmaven.repo.local=/data/mvn/repo verify' } },JRE7Test: { docker.image('java:jre-7').inside('-v /data:/data') { sh 'mvn -Dmaven.repo.local=/data/mvn/repo verify' } }, failFast: true )}
75. Pipeline - Docker Example node('docker-cloud') { stage 'Build Docker Image' dir('target') { mobileDepositApiImage = docker.build "beedemo/mobile-deposit-api:$ {dockerTag}" } stage 'Publish Docker Image' withDockerRegistry(registry: [credentialsId: 'docker-hub-beedemo']) { mobileDepositApiImage.push() } }