如何設計一套具備 Container 容器化技術的 CI/CD 平台?

我相信大家在團隊內一定都有使用 CI/CD 工具，像是 Jenkins、GitLab CI、Travis CI、Circle CI 等等，這些工具都是非常好用的，但是如果想要在自己的環境中建置一套 CI/CD 平台，你會怎麼做呢？這議程會帶您深入了解 Drone CI/CD 開源工具，並且透過實際的 Demo 來說明如何設計一套具備 Container 容器化技術的 CI/CD 平台。藉由 Drone 原始碼的解析，讓您了解 Drone 的運作原理，並且透過 Drone 的 Plugin 機制，讓您可以自行開發 Drone 的 Plugin，讓 Drone 可以更加符合您的需求。

https://cloudsummit.ithome.com.tw/2024/session-page/2653

Transcript

1. 如何設計⼀套具備 Container 容器化技術的 CI/CD 平台 Bo-Yi Wu @ Mediatek https://blog.wu-boy.com/ 2024/07/03
2. (JU)VC
   "DUJPOT +FOLJOT (JU-BC
   $*
3. What is Drone?
4. Drone 特性 • 使⽤ Go 語⾔編寫，所有組件都運⾏在 Docker 容器中 • ⽀持多種操作系統和架構（Linux、Windows、Mac） • 可擴展架構，具⾃動縮放功能，根據需求管理減少雲成本 • 可選擇安裝在本地或作為具有附加功能的 SaaS 產品使⽤ • 可使⽤任何語⾔編寫插件 • 管道使⽤簡單語法的 YAML 格式定義。 • 開源
5. Architecture
7.      
8. Drone Runner
10. 4UBSU
    UIF
    1PMMFS
    "HFOU
11. poller := &poller.Poller{ Client: cli, Dispatch: runner.Run, Filter: &client.Filter{ Kind: resource.Kind, Type: resource.Type, OS: config.Platform.OS, Arch: config.Platform.Arch, Variant: config.Platform.Variant, Kernel: config.Platform.Kernel, Labels: config.Runner.Labels, }, } ࠜᎦෆಉܥ౷㐫ଶ
    ܾఆࣥߦಛఆ೚຿ ՄҎઃఆ
    (16
    $16
    ෆಉओػ
12. func (p *Poller) Poll(ctx context.Context, n int) { var wg sync.WaitGroup for i := 0; i < n; i++ { wg.Add(1) go func(i int) { for { select { case <-ctx.Done(): wg.Done() return default: p.poll(ctx, i+1) } } }(i) } wg.Wait() } ։啟ଟݸ
    1PMMFS
    ฏߦ႔ཧ ։࢝ࣥߦ೚຿
13. 'FUDI
    UIF
    /FX
    5BTL
14.  ॳ࢝Խ࿈ઢ။۰౸ॄኄ໰୊
15. ೗Ռ༗
    
    ݸ
    1PMMFS
    "HFOU
    ॳ࢝Խ
    
    
    
    
    
    ݸ
    $MJFOU
    %%04
16. func (p *Poller) Poll(ctx context.Context, n int) { var wg sync.WaitGroup for i := 0; i < n; i++ { wg.Add(1) go func(i int) { for { select { case <-ctx.Done(): wg.Done() return default: p.poll(ctx, i+1) } } }(i) } wg.Wait() } ։啟ଟݸ
    1PMMFS
    ฏߦ႔ཧ ։࢝ࣥߦ೚຿ ໰୊
17. ೗Ռ༗
    
    ݸ
    1PMMFS
    "HFOU
    ॳ࢝Խ
    
    
    
    
    
    ݸ
    $MJFOU
    %%04 㑌ݸॳ࢝Խ
    1PMMFS
    ޙᬋػఀ಴
    YYY
    NT
    ՄҎݮ᫊
    %SPOF
    4FSWFS
    ෛ፦
    
18. func (p *Poller) Poll(ctx context.Context, n int) { var wg sync.WaitGroup for i := 0; i < n; i++ { wg.Add(1) go func(i int) { for { select { case <-ctx.Done(): wg.Done() return default: p.poll(ctx, i+1) } } }(i) } wg.Wait() } ։啟ଟݸ
    1PMMFS
    ฏߦ႔ཧ ։࢝ࣥߦ೚຿ ṜᬑՄҎᬋػఀ಴
    YYY
    NT
19. ၊ઃਅ༗
    
    ݸ
    1PMMFS
    ಉ࣌᪑
    4FSWFS
    ᩇऔधཁࣥߦత೚຿
    ᙛલᔒ༗೚Կ೚຿
20. func (p *Poller) Poll(ctx context.Context, n int) { var wg sync.WaitGroup for i := 0; i < n; i++ { wg.Add(1) go func(i int) { for { select { case <-ctx.Done(): wg.Done() return default: p.poll(ctx, i+1) } } }(i) } wg.Wait() } ։啟ଟݸ
    1PMMFS
    ฏߦ႔ཧ ։࢝ࣥߦ೚຿ ໰୊
21. ၊ઃਅ༗
    
    ݸ
    1PMMFS
    ಉ࣌᪑
    %SPOF
    4FSWFS
    ᩇऔधཁࣥߦత೚຿ ᪑೭લղ๏Ұᒬɼࡏ
    1PMMFS
    ׬੒ޙҰᒬ
    ᬋػఀ಴
    YY
    ඵݮগ
    4FSWFS
    ෛ፦
22. func (p *Poller) Poll(ctx context.Context, n int) { var wg sync.WaitGroup for i := 0; i < n; i++ { wg.Add(1) go func(i int) { for { select { case <-ctx.Done(): wg.Done() return default: p.poll(ctx, i+1) } } }(i) } wg.Wait() } ։啟ଟݸ
    1PMMFS
    ฏߦ႔ཧ ։࢝ࣥߦ೚຿ ṜᬑՄҎᬋػఀ಴
    YY
    ඵ
23. func (p *Poller) Poll(ctx context.Context, n int) { var wg sync.WaitGroup for i := 0; i < n; i++ { wg.Add(1) go func(i int) { for { select { case <-ctx.Done(): wg.Done() return default: p.poll(ctx, i+1) } } }(i) } wg.Wait() } ։啟ଟݸ
    1PMMFS
    ฏߦ႔ཧ ։࢝ࣥߦ೚຿ ṜᬑՄҎᬋػఀ಴
    YY
    ඵ ࠑղ๏။Ҿᚙॄኄ໰୊ʁ
24. 4MFFQ
    T 4MFFQ
    T 4MFFQ
    T େྔ
    5BTL
    ༙ೖ
25. Drone Server 如何解決此問題
26. ೗Կղܾେྔత
    "HFOU
    查ᨑ໨લ֘ࣥߦత೚຿
27. How to handle Agent request? • 建立 Timeout 為 30 秒的長連線 (Agent) • 使⽤ Go 語⾔ Map 建立所有 Agent 表列 • 使⽤ Channel 將項⽬分配到 Agent
28. func (s *Server) handleRequest(w http.ResponseWriter, r *http.Request) { ctx := r.Context() ctx, cancel := context.WithTimeout(ctx, defaultTimeout) defer cancel() in := &requestRequest{} err := json.NewDecoder(r.Body).Decode(in) if err != nil { writeBadRequest(w, err) return } stage, err := s.manager.Request(ctx, in.Request) if err != nil { writeError(w, err) return } json.NewEncoder(w).Encode(stage) } 
    
    UJNF4FDPOE ݐཱ
    
    ඵత௕࿈ઢ
29. q.Lock() q.workers[w] = struct{}{} q.Unlock() select { case q.ready <- struct{}{}: default: } select { case <-ctx.Done(): q.Lock() delete(q.workers, w) q.Unlock() return nil, ctx.Err() case b := <-w.channel: return b, nil } ࣥߦҰ࣍查ᨑ
    %# "HFOU
    ࢿ㘤์౸
    XPSLFS
    㚎 ᩇऔ
    4UBHF
    ࢿ㘤 ௒ա
    
    ඵ௚઀Ҡআ
    XPSLFS
30. q.Lock() count := len(q.workers) pause := q.paused q.Unlock() if pause { return nil } if count == 0 { return nil } items, err := q.store.ListIncomplete(ctx) if err != nil { return err } ᩇऔॴ༗ঘະࣥߦత೚຿ Ꭴग़ॴ༗ະࣥߦత೚຿ฒ෼഑౸ಛఆత
    "HFOU
31. Drone Server 更好的解法?
33.  㑌ݸ
    "HFOU
    ୞။༗Ұᑍ࿈ઢ
     ՄҎಈଶௐ੔
    8PSLFS
    Ꮠྔ
36.  ֬อ୞༗ᄸҰ̣PEF
    ՄҎ፤౸
37.  ᩇऔ
    1JQFMJOF
    㑊ᱜ
38.  ؂ᡒ
    $BODFM
    ࣄ݅
39. ctxdone, cancel := context.WithCancel(ctx) defer cancel() timeout := time.Duration(data.Repo.Timeout) * time.Minute ctxtimeout, cancel := context.WithTimeout(ctxdone, timeout) defer cancel() ctxcancel, cancel := context.WithCancel(ctxtimeout) defer cancel() go func() { done, _ := s.Client.Watch(ctxdone, data.Build.ID) if done { cancel() log.Debugln("received cancellation") } else { log.Debugln("done listening for cancellations") } }()
40. ctxdone, cancel := context.WithCancel(ctx) defer cancel() timeout := time.Duration(data.Repo.Timeout) * time.Minute ctxtimeout, cancel := context.WithTimeout(ctxdone, timeout) defer cancel() ctxcancel, cancel := context.WithCancel(ctxtimeout) defer cancel() go func() { done, _ := s.Client.Watch(ctxdone, data.Build.ID) if done { cancel() log.Debugln("received cancellation") } else { log.Debugln("done listening for cancellations") } }() 
41. ctxdone, cancel := context.WithCancel(ctx) defer cancel() timeout := time.Duration(data.Repo.Timeout) * time.Minute ctxtimeout, cancel := context.WithTimeout(ctxdone, timeout) defer cancel() ctxcancel, cancel := context.WithCancel(ctxtimeout) defer cancel() go func() { done, _ := s.Client.Watch(ctxdone, data.Build.ID) if done { cancel() log.Debugln("received cancellation") } else { log.Debugln("done listening for cancellations") } }() 
42. ctxdone, cancel := context.WithCancel(ctx) defer cancel() timeout := time.Duration(data.Repo.Timeout) * time.Minute ctxtimeout, cancel := context.WithTimeout(ctxdone, timeout) defer cancel() ctxcancel, cancel := context.WithCancel(ctxtimeout) defer cancel() go func() { done, _ := s.Client.Watch(ctxdone, data.Build.ID) if done { cancel() log.Debugln("received cancellation") } else { log.Debugln("done listening for cancellations") } }() 
43. ctxdone, cancel := context.WithCancel(ctx) defer cancel() timeout := time.Duration(data.Repo.Timeout) * time.Minute ctxtimeout, cancel := context.WithTimeout(ctxdone, timeout) defer cancel() ctxcancel, cancel := context.WithCancel(ctxtimeout) defer cancel() go func() { done, _ := s.Client.Watch(ctxdone, data.Build.ID) if done { cancel() log.Debugln("received cancellation") } else { log.Debugln("done listening for cancellations") } }() 
44. ctxdone, cancel := context.WithCancel(ctx) defer cancel() timeout := time.Duration(data.Repo.Timeout) * time.Minute ctxtimeout, cancel := context.WithTimeout(ctxdone, timeout) defer cancel() ctxcancel, cancel := context.WithCancel(ctxtimeout) defer cancel() go func() { done, _ := s.Client.Watch(ctxdone, data.Build.ID) if done { cancel() log.Debugln("received cancellation") } else { log.Debugln("done listening for cancellations") } }()  3FTQPOTF
    4UBUVT
    $PEF
    
    
     3FTQPOTF
    4UBUVT
    $PEF
    
    
45. %JSFDUFE
    "DZDMJD
    (SBQI
46. %"(
    &YBNQMF
47. kind: pipeline type: docker name: default steps: - name: test image: golang:1.22 commands: - go build - go test -v - name: notify image: plugins/slack settings: channel: dev webhook: from_secret: endpoint ٬੡ԽჩᏐ
    */165@
48. END