Drone CI/CD 自動化測試及部署

## 課程大綱

1. Drone 基礎認識
2. Drone 安裝方式
3. Drone 基本用法與指令介紹
4. Drone 外掛介紹與外掛撰寫

## 課程簡述

1. 用 Go 語言所打造的 Drone 輕量級容器持續交付平台，Drone 是一套基於 Docker 容器技術的持續交付平台
2. 每個 Build 都在一個短暫的 Docker 容器中執行，讓開發人員可以完全控制他們的建置環境並保證隔離互不干擾
3. Drone 非常容易安裝及維護，並且提供強大的插件（plugin）系統，也可以讓開發者輕易完成個人插件
4. 如果您正在煩惱該選擇 Jenkins 或 GitLab CI 等工具，建議您來嘗試看看本次課程介紹之輕量級的 Drone
5. 本課程會帶您深入了解由 Go 語言所撰寫的 Drone 架構及如何輕易整合原有的 Git 服務（像是 Github、Bitbucket、Gitea 或 GitLab）來快速部署及測試軟體品質

Transcript

1. Drone CI/CD ⾃動化測試及部署
2. About me • Software Engineer in Mediatek • Member of Drone CI/CD Platform • Member of Gitea Platform • Member of Gin Golang Framework • Teacher of Udemy Platform: Golang + Drone https://blog.wu-boy.com
3. 課程⼤綱 • Docker 基礎介紹 • CI/CD 服務簡介 • Drone 架構介紹 • Drone 基礎使⽤ • Drone 外掛撰寫
4. Docker 容器介紹
5. ၚ౷
   7.
   㐘ٖػՍߏ
6. VM 虛擬機好處 • 不同的作業系統環境 • 善⽤資源分配 (CPU, RAM, Storage) • 透過 Hypervisor 管理 VM 快照
7. $POUBJOFS
   ༰ثՍߏ
8. Container 容器好處 • 容器檔案非常⼩ (Ubuntu 才⼗幾 MB) • 快速開啟 (不到⼀秒就可以起⼀個 Ubuntu) • 透過 Docker fi le 客製化容器內容 • 快速實踐 Deployment + Operation
9. docker 基礎指令 • docker ps • 列出正在跑的容器 • docker images • 列出已經下載的 Images • docker image prune -a -f • 清除沒在使⽤的 Images (空間如果不夠)
10. • docker container prune -f • 清除沒在跑的容器 • docker login -u xxxx • 登入 docker registry (GitLab 內建, 預設 DockerHub) • docker run -ti ubuntu /bin/bash • 快速建立 ubuntu 容器並且登入 Bash Shell • ⽤於測試及除錯
11. 建立 Docker 服務 • docker run -d -p 8081:80 nginx • docker run -d -p 8082:80 nginx • docker run -d -p 8083:80 nginx • -p port 對應關係 (host port : container port) • -d 丟到背景執⾏ • —name 容器命名 • -e 環境變數 • -v 掛載硬碟
12. 進入已存在容器 • docker exec -ti name /bin/bash • docker ps 可以知道正在跑的容器 • 除錯常⽤
13. 練習 • 在本機端架設底下服務 • redis (port 6380:6379) • mysql (port 3307:3306) • postgres (port 5433:5432) • ⽤ Go 或 Node 測試連線
14. 練習 • 架設 Go Module Proxy • https://github.com/gomods/athens • export GOPROXY=http://localhost:3010 • 掛載 local 端空間到 athens
15. 建立容器
16. 使⽤ Docker fi le
17. FROM alpine:3.15 LABEL maintainer="Bo-Yi Wu <appleboy.tw@gmail.com>" RUN apk add - - no - cache ca - certif i cates & & \ rm - rf /var/cache/apk / * COPY release/linux/amd64/agent /bin/ ENTRYPOINT [ "/bin/agent" ]
18. 編譯容器並上傳 • docker build -t appleboy/server:tag -f path . • -t 組織名稱/容器名稱 • -f Docker fi le 檔案路徑 • tag: 版本 (⽤來 deploy production) • docker push host:port/appleboy/server • host: docker registry host (預設上傳到 docker hub) • port: docker registry port
19. 多個服務串接
20. 使⽤ docker-compose.yml
21. version: '3.1' services: prometheus: image: prom/prometheus:v2.31.1 volumes: - ./prometheus/:/etc/prometheus/ - ./data/prometheus:/prometheus command: - ' - - conf i g.f i le=/etc/prometheus/prometheus.yml' - ' - - storage.tsdb.path=/prometheus' - ' - - web.console.libraries=/usr/share/prometheus/console_libraries' - ' - - web.console.templates=/usr/share/prometheus/consoles' - ' - - storage.tsdb.retention.time=365d' restart: always grafana: image: grafana/grafana:8.2.6 depends_on: - prometheus volumes: - ./data/grafana:/var/lib/grafana - ./grafana/provisioning/:/etc/grafana/provisioning/ env_f i le: - ./grafana/conf i g.monitoring restart: always
22. docker-compose 指令 • docker-compose up -d [service_name] • -d 在背景執⾏ • docker-compose down -v • -v 移除 volume 資料 • docker-compose logs -f [service_name] • 看單⼀服務 Logs • docker-compose pull • 更新全部 Image (每次 deploy 都需要做此步驟) • docker-compose exec db /bin/bash • 進入容器看 DB 資料
23. 練習 • 將 Go 服務搭配 redis + (mysql or postgres) • redis • mysql • postgres • ⽤ docker-compose 串起上⾯服務
24. 部署流程 GitHub Flow vs Git Flow IUUQTCMPHXVCPZDPNHJUIVC fl PXWTHJU fl PX
25. (JU)VC
    'MPX 14 Develop Git Push Git Tag Develop Git Push Git Tag Testing Deploy Deploy Deploy Production Staging Production Testing Deploy Staging
26. GitHub Flow + Git Flow in opensource IUUQTHJUIVCDPNHPHJUFBHJUFB
27. 如何⾃動化上⾯的步驟?
28. 為什麼我不選 Jenkins?
29. Why? • Complicated project setting • Write the plugin (Java language) • Maintenance? • Learning Curve? • Grow your team?
30. 為什麼我不選 GitLab CI?
31. Why? • GitLab Only • Not Extensible
32. Drone CI/CD 介紹
33. Drone CI • Container native CI/CD platform • Easy to install & maintain • Isolate builds • Simple YAML Con fi guration • Integrates with several VCS Providers • Rich set of of fi cial plugins (any container can be a plugin) • Execute locally with simple command (drone exec) • open source (https://github.com/harness/drone)
34. Drone CI/CD 平台真的免費嗎 IUUQTCMPHXVCPZDPNESPOFMJDFOTF
35. Everything is a Docker Container
36. Project CI/CD Flow HJU
    DMPOF UFTUJOH EFQMPZ OPUJGZ
37. Drone CI Infrastructure Runner Server Step 1 git clone Step 2 make build Step 3 deploy app work space extra service Runner
38. 如何安裝 Drone + GitHub
39. 申請 GitHub Application
40. Apply GitHub Application https: / / github.com/settings/developers
41. https: / / github.com/settings/applications/new
42. Drone Server and Runner In docker-compose.yml
43. version: '3' services: drone - server: image: drone/drone:2 ports: - 8081 : 80 volumes: - ./:/data restart: always environment: - DRONE_SERVER_HOST=${DRONE_SERVER_HOST} - DRONE_SERVER_PROTO=${DRONE_SERVER_PROTO} - DRONE_RPC_SECRET=${DRONE_RPC_SECRET} # GitHub Conf i g - DRONE_GITHUB_SERVER=https: / / github.com - DRONE_GITHUB_CLIENT_ID=${DRONE_GITHUB_CLIENT_ID} - DRONE_GITHUB_CLIENT_SECRET=${DRONE_GITHUB_CLIENT_SECRET} - DRONE_LOGS_PRETTY=true - DRONE_LOGS_COLOR=true $SFBUF
    EPDLFSDPNQPTFZNM
44. DRONE_SERVER_HOST= DRONE_SERVER_PROTO= DRONE_RPC_SECRET= DRONE_GITHUB_CLIENT_ID= DRONE_GITHUB_CLIENT_SECRET= $SFBUF
    FOW
    fi MF
45. drone - runner: image: drone/drone - runner - docker:1 restart: always depends_on: - drone - server ports: - 3000 : 3000 volumes: - /var/run/docker.sock:/var/run/docker.sock environment: - DRONE_RPC_HOST=${DRONE_SERVER_HOST} - DRONE_RPC_PROTO=${DRONE_SERVER_PROTO} - DRONE_RPC_SECRET=${DRONE_RPC_SECRET} - DRONE_RUNNER_CAPACITY=2 - DRONE_RUNNER_NAME=my - f i rst - runner
46. Drone Web UI Introduction
51. 撰寫 .drone.yml 檔案
52. kind: pipeline type: docker name: default steps: - name: greeting image: alpine commands: - echo hello - echo world
53. kind: pipeline type: docker name: linux_amd64 steps: - name: build image: golang:1.17 commands: - make build_linux_amd64
54. build_linux_amd64 : CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \ - v - a - o release/linux/amd64/helloworld
55. build_linux_amd64 : CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \ - v - a - o release/linux/amd64/helloworld
56. - - - kind: pipeline type: docker name: linux_amd64 steps: - name: build image: golang:1.17 environment: GOOS : linux GOARCH : amd64 CGO_ENABLED : 0 commands: - make build_linux_amd64
57. build: go build - v - a - o \ release/${GOOS}/${GOARCH}/helloworld
58. kind: pipeline type: docker name: linux_amd64 steps: - name: build image: golang:1.17 commands: - make build_linux_amd64 - - - kind: pipeline type: docker name: linux_i386 steps: - name: build image: golang:1.17 commands: - make build_linux_i386 .VMUJQMF
    1JQFMJOF
59. - - - kind: pipeline type: docker name: linux_i386 steps: - name: build image: golang:1.17 environment: GOOS : linux GOARCH : 386 CGO_ENABLED : 0 commands: - make build_linux_i386 $VTUPN
    &OWJSPONFOU
60. - - - kind: pipeline type: docker name: linux_i386 environment: GOOS : linux GOARCH : 386 CGO_ENABLED : 0 steps: - name: build image: golang:1.17 commands: - make build_linux_i386 (MPCBM
    &OWJSPONFOU
61. Routing 將特定任務丟到特定主機 (GPU or High Memory)
62. kind: pipeline type: docker name: default steps: - name: build image: golang commands: - go build - go test node: keyA : valueA keyB : valueB %30/&@36//&3@-"#&-4LFZ"WBMVF" LFZ#WBMVF#
63. Parallelism 平⾏處理
64. kind: pipeline type: docker name: default steps: - name: backend image: golang commands: - go build - go test - name: frontend image: node commands: - npm install - npm test - name: notify image: plugins/slack depends_on: - frontend - backend
65. Conditions and Triggers
66. kind: pipeline type: docker name: default steps: - name: build image: golang commands: - go build - go test when: branch: - master - feature / * kind: pipeline type: docker name: default steps: - name: build image: golang commands: - go build - go test trigger: branch: - master
67. Volumes
68. kind: pipeline type: docker name: default steps: - name: test image: golang volumes: - name: cache path: /go commands: - go get - go test - name: build image: golang volumes: - name: cache path: /go commands: - go build volumes: - name: cache temp: {}
69. 練習 • 任何程式語⾔進⾏ Lint + Testing + Build • Go 語⾔使⽤ golangci-lint • 測試 go test -v -cover • 編譯 go build -o release/linux/amd64/out
70. Plugin IUUQQMVHJOTESPOFJP
71. - name: publish pull: always image: plugins/docker:linux - amd64 settings: auto_tag: true auto_tag_suff i x: linux - amd64 cache_from: appleboy/gorush daemon_off: false dockerf i le: docker/Dockerf i le.linux.amd64 password: from_secret: docker_password repo: appleboy/gorush username: from_secret: docker_username when: event: exclude: - pull_request
72. - name: publish pull: always image: plugins/docker:linux - amd64 settings: auto_tag: true auto_tag_suff i x: linux - amd64 cache_from: appleboy/gorush daemon_off: false dockerf i le: docker/Dockerf i le.linux.amd64 password: from_secret: docker_password repo: appleboy/gorush username: from_secret: docker_username when: event: exclude: - pull_request
74. Plugin Input
75. - name: publish pull: always image: plugins/docker:linux - amd64 settings: auto_tag: true auto_tag_suff i x: linux - amd64 cache_from: appleboy/gorush daemon_off: false dockerf i le: docker/Dockerf i le.linux.amd64 password: from_secret: docker_password repo: appleboy/gorush username: from_secret: docker_username when: event: exclude: - pull_request
76. PLUGIN_USERNAME=appleboy PLUGIN_PASSWORD=password PLUGIN_REPO=appleboy/gorush PLUGIN_TAGS=1.0.0,1.0
77. PLUGIN_USERNAME=appleboy PLUGIN_PASSWORD=password PLUGIN_REPO=appleboy/gorush PLUGIN_TAGS=1.0.0,1.0
78. 如何製作 Plugins
79. Go Plugin vs Bash Plugin (Any language you want)
80. Bash Plugin
81. kind: pipeline type: docker name: default steps: - name: webhook image: acme/webhook settings: url: http: / / hook.acme.com method: post body: | hello world
82. # ! /bin/sh curl \ -X ${PLUGIN_METHOD} \ - d ${PLUGIN_BODY} \ ${PLUGIN_URL}
83. FROM alpine:3.14 ADD script.sh /bin/ RUN chmod + x /bin/script.sh RUN apk -Uuv add curl ca - certif i cates ENTRYPOINT /bin/script.sh
84. $ docker build -t acme/webhook . $ docker push acme/webhook $ docker run --rm \ -e PLUGIN_METHOD=post \ -e PLUGIN_URL=http://hook.acme.com \ -e PLUGIN_BODY=hello \ acme/webhook ฤᩄ
85. $ docker build -t acme/webhook . $ docker push acme/webhook $ docker run --rm \ -e PLUGIN_METHOD=post \ -e PLUGIN_URL=http://hook.acme.com \ -e PLUGIN_BODY=hello \ acme/webhook ଌࢼ
86. Go Plugin
87. package main import ( "net/http" "os" "strings" ) func main() { body : = strings.NewReader( os.Getenv("PLUGIN_BODY"), ) _, err : = http.NewRequest( os.Getenv("PLUGIN_METHOD"), os.Getenv("PLUGIN_URL"), body, ) if err ! = nil { os.Exit(1) } } GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o webhook
88. FROM alpine:3.14 ADD webhook /bin/ RUN apk -Uuv add ca - certif i cates ENTRYPOINT /bin/webhook
89. 練習 • 任何程式語⾔進⾏ Plugin 撰寫 • 申請 dockerhub 帳號密碼 • 透過 Drone ⾃動打包 Image 並上傳
90. Deploy to Production Server via SSH Command
91. - name: ssh commands image: appleboy/drone - ssh settings: host: foo.com username: root key: from_secret: ssh_key passphrase: 1234 port: 22 script: - mkdir abc/def/efg - echo "you can't see the steps."
92. Promotions ⼿動部署
93. kind: pipeline type: docker name: default steps: - name: test image: node commands: - npm install - npm run test - npm run bundle - name: deploy image: appleboy/drone - ssh settings: . . . trigger: event: - promote target: - production kind: pipeline type: docker name: deploy steps: - name: test image: node commands: - npm install - npm run test - npm run bundle - name: deploy image: appleboy/drone - ssh settings: . . . when: event: - promote target: - production
94. $ drone build promote octocat/hello-world 42 staging $ drone build promote <repo> <number> <environment> $ drone build promote octocat/hello-world 42 production
95. Thanks