Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
GolangでDockerベースのCIを作る
Search
Shunsuke Maeda
May 18, 2019
Technology
3
3.7k
GolangでDockerベースのCIを作る
Golang を使って Dockerベース のCIを作るお話です。
Shunsuke Maeda
May 18, 2019
Tweet
Share
More Decks by Shunsuke Maeda
See All by Shunsuke Maeda
静的解析ツール detekt で任意の条件で警告させる
duck8823
1
1.4k
GitHub と連携する CI を作る
duck8823
3
2.6k
Other Decks in Technology
See All in Technology
Beyond Kaniko: Navigating Unprivileged Container Image Creation
f30
0
130
Lufthansa ®️ USA Contact Numbers: Complete 2025 Support Guide
lufthanahelpsupport
0
190
ネットワーク保護はどう変わるのか?re:Inforce 2025最新アップデート解説
tokushun
0
200
Core Audio tapを使ったリアルタイム音声処理のお話
yuta0306
0
190
AI時代の開発生産性を加速させるアーキテクチャ設計
plaidtech
PRO
3
150
FOSS4G 2025 KANSAI QGISで点群データをいろいろしてみた
kou_kita
0
400
B2C&B2B&社内向けサービスを抱える開発組織におけるサービス価値を最大化するイニシアチブ管理
belongadmin
1
6.9k
SaaS型なのに自由度の高い本格CMSでサイト構築と運用のコスパ&タイパUP! MovableType.net の便利機能とユーザー事例のご紹介
masakah
0
110
Sansanのデータプロダクトマネジメントのアプローチ
sansantech
PRO
0
150
KiCadでPad on Viaの基板作ってみた
iotengineer22
0
300
united airlines ™®️ USA Contact Numbers: Complete 2025 Support Guide
flyunitedhelp
1
300
マーケットプレイス版Oracle WebCenter Content For OCI
oracle4engineer
PRO
3
960
Featured
See All Featured
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
357
30k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
Automating Front-end Workflow
addyosmani
1370
200k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Transcript
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ ɹ GoConference 2019 Spring
2019.05.18 [SAT] Shunsuke Maeda (@duck8823)
ࣗݾհ ✓ Shunsuke Maeda ✓ @duck8823 ✓ ॴଐ ✓גࣜձࣾΤεɾΤϜɾΤε ✓EarthCampusגࣜձࣾʢ෭ʣ
2/43
CI͠ΜͲ͍ 3/43
CI͠ΜͲ͍ ! ✓CIΛߏங͍ͨ͠ ci := ci.New() for ci.IsBroken { //
CI͕յΕͯͨΒ fix(ci) // ϩʔΧϧͰ͚͢Ͳಈ࡞֬ೝͰ͖ͳ͍ͷͰ git.Commit() // ίϛοτ ͯ͠ git.Push() // ϓογϡ ͯ͠ ci.Run() // CI ಈ͔ͯ͠ΈΔ } 4/43
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ duck8823/duci 5/43
duci ͷಛ ✓DockerfileͰఆٛ ✓ ֶशίετ͕͍ ✓ ϩʔΧϧͰ࣮ߦ ✓GitHub ʹରԠ ✓
Push / PR্ͷίϝϯτ Ͱ࣮ߦ ✓ ίϛοτεςʔλεΛ࡞ 6/43
ֶशίετ ✓Dockerfile ͷΈ FROM golang:1.12.4-alpine # ϕʔεΠϝʔδ RUN apk --update
add --no-cache ... # ඞཁͳύοέʔδͷΠϯετʔϧ WORKDIR /workdir COPY . . ENTRYPOINT ["make"] # λεΫϥϯφʔͷར༻Λਪ CMD ["test"] # σϑΥϧτͷλεΫ 7/43
ֶशίετ ✓DockerͷࣝͰΩϟογϡԽ FROM golang:1.12.4-alpine RUN apk --update add --no-cache ...
WORKDIR /workdir COPY go.mod . COPY go.sum . RUN go mod download COPY . . ENTRYPOINT ["make"] CMD ["test"] 8/43
ϩʔΧϧͰ࠶ݱ ✓ίϚϯυΛ༻ҙ ✓ϩʔΧϧͰࢼ͔ͯ͠Β commit & push Ͱ͖Δ duci run INFO[14/May/2019
09:25:57.353] Step 1/9 : FROM openjdk:11 as Build INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> 0aa10063a184 INFO[14/May/2019 09:25:57.353] Step 2/9 : WORKDIR /workdir INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> Using cache INFO[14/May/2019 09:25:57.353] ---> 2dcc968d8994 ... 9/43
CIΛ࡞Δ 10/43
Continuous Integration 11/43
Continuous Integration ✓ఆظతʹ ✓ϏϧυςετΛ࣮ߦ ͢Δ͜ͱͰ ✓ૣظʹ ϑΟʔυόοΫ Λಘͯ όά͕ຊ൪ڥʹࠞೖ͢ΔͷΛ͙ 12/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
13/43
duci ʹ͓͚Δߏ ✓࣮ߦλΠϛϯά ✓ GitHub Webhooks ✓࣮ߦڥ ✓ Dockerίϯςφ ✓࣮ߦ݁Ռ
✓ ϩάग़ྗ / GitHub ίϛοτεςʔλε 14/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
15/43
࣮ߦλΠϛϯά ✓GitHub Webhooks https://github.com/<owner>/<repo>/settings/hooks ✓ ҙͷλΠϛϯάͰ Payload Λඈ͢ 16/43
PayloadΛड͚औΔ ✓GitHub ͔ΒඈΜͰ͘Δ Payload JSON Λड͚औΔඞཁ͕͋Δ import "net/http" func JobHandler(w
http.ResponseWriter, r *http.Request) { // ֤ΠϕϯτʹରԠͨ͠PayloadͷύʔεॲཧͱCIδϣϒͷ࣮ߦ } func main() { http.HandleFunc("/", JobHandler) http.ListenAndServe(":8080", nil) } 17/43
GitHub ͷ Webhooks ✓Event ͱ Action ͷΈ߹ΘͤͰλΠϛϯά੍ޚ ✓Event ✓ Webhooksͷछྨ
(Header: X-GitHub-Event) ྫ. push, issue_comment, pull_request ✓ GitHubͰઃఆՄೳ ✓Action ✓ Eventຖͷࡉ͔͍छྨ (JSON payload) ྫ. created, opened, synchronize 18/43
Eventͷऔಘ ✓ϦΫΤετͷϔομʔ X-GitHub-Event ͔ΒΠϕϯτΛऔಘ ✓Πϕϯτຖʹ Payload ͕ҟͳΔ func JobHandler(w http.ResponseWriter,
r *http.Request) { event := r.Header.Get("X-GitHub-Event") switch event { case "push": // https://developer.github.com/v3/activity/events/types/#pushevent case "issue_comment": // https://developer.github.com/v3/activity/events/types/#issuecommentevent case "pull_request": // https://developer.github.com/v3/activity/events/types/#pullrequestevent default: http.Error(w, fmt.Sprintf("Bad event type: %s", event), http.StatusBadRequest) } } 19/43
Payload JSON ͷऔΓѻ͍ ✓encoding/json ͰσίʔυͰ͖Δ ✓Payloadͷܕ github.com/google/go-github import ( "encoding/json"
"github.com/google/go-github/github" ) func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "push": event := &github.PushEvent{} err := json.NewDecoder(r.Body).Decode(event) ... } } 20/43
Forkͨ͠ϦϙδτϦ͔Βͷ Pull Request ✓push ΠϕϯτඈΜͰ͜ͳ͍ ✓ pull_request Πϕϯτͷ synchronize ΞΫγϣϯ
func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "pull_request": event := &github.PullRequestEvent{} json.NewDecoder(r.Body).Decode(event) switch event.GetAction() { case "synchronize", "open": ... } } } 21/43
Pull Request ͷίϝϯτ import ( "github.com/google/go-github/github" "golang.org/x/oauth2" ) func JobHandler(w
http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "issue_comment": event := &github.IssueComment{} json.NewDecoder(r.Body).Decode(event) ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "access token"}) tc := oauth2.NewClient(context.Background(), ts) cli := github.NewClient(tc) pr := cli.PullRequests.Get( context.Background(), ownerName, repoName, event.GetIssue().GetNumber(), ) ... } } 22/43
ඇಉظͰϨεϙϯεΛฦ͢ ✓δϣϒ͕͔͔࣌ؒΔ => ඇಉظʹ࣮ߦͯ͠ϨεϙϯεΛฦ͢ func JobHandler(w http.ResponseWriter, r *http.Request) {
if (invalidRequest(r)) { http.Error(w, r.Error(), http.StatusBadRequest) return } go func() { // CIδϣϒͷ࣮ߦ }() w.WriteHeader(http.StatusOK) } 23/43
࣮ߦλΠϛϯά (·ͱΊ) ✓ϦΫΤετड => net/http ✓Payload ͷܕ => ϥΠϒϥϦͷར༻ ྫ.
GitHub Webhooks: github.com/google/go-github ✓ GitHub Event ͱ Action ͰλΠϛϯάΛ੍ޚ ✓ඇಉظͰϨεϙϯεΛฦ͢ => goroutine 24/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
25/43
࣮ߦڥ ✓Dockerίϯςφ ✓ ϗετΛԚ͞ͳ͍ ✓ ϩʔΧϧ / CI Ͱͷಈ࡞ʹ࠶ݱੑΛͱΓ͍͢ ✓github.com/docker/docker
Λར༻ 26/43
github.com/docker/docker ✓ϦϙδτϦ moby/moby ✓Docker Daemon ͷରԠAPIόʔδϣϯ ɹ- Docker for Mac
1.39 ✓ Moby ͷ master 1.41 docker version Server: Docker Engine - Community Engine: Version: 18.09.2 API version: 1.39 (minimum version 1.12) ... 27/43
DockerΠϝʔδ/ίϯςφΛѻ͏ import "github.com/docker/docker/client" cli, _ := client.NewClientWithOpts(client.FromEnv) ✓ײతͳAPI ✓ cli#ImageBuild
= docker build ✓ cli#ContainerCreate = docker create ✓ cli#ContainerStart = docker start 28/43
DockerΠϝʔδͷϏϧυ ✓ίϯςΩετʢσΟϨΫτϦʣΛ tarball ͷܗࣜʹ͢Δඞཁ͕͋Δ ✓cli#ImageBuild ✓ ඇಉظʹ࣮ߦ͞ΕΔ ✓ resp.Body Λ
EOF ·ͰಡΜͰऴྃͪ opts := types.ImageBuildOptions{ Tags: []string{"tag"} } resp, _ := cli.ImageBuild(ctx, tarball("/workdir"), opts) ioutil.ReadAll(resp.Body) 29/43
ίϯςφͷ࣮ߦ ✓ίϯςφͷ࡞ cli#ContainerCreate ✓cli#ContainerStart ඇಉظ ✓ cli#ContainerWait cli#ContainerLogs ͰδϣϒͷใΛऔಘ
sopts := types.ContainerStartOptions{} cli.ContainerStart(context.Background(), "containerId", sopts) code, _ := cli.ContainerWait(context.Background(), "containerId") if code != 0 { // exit code ͕ 0Ҏ֎ => δϣϒͷࣦഊ } lopts := types.ContainerLogsOptions{ ShowStdout: true, ShowStderr: true } log _ := cli.ContainerLogs(context.Background(), "containerId", lopts) 30/43
δϣϒͷฒྻͱλΠϜΞτ ✓ϦΫΤετຖʹ goroutine Ͱ࣮ߦ ✓ δϣϒΛ͍࣮ͭ͘ߦͯ͠͠·͏ ✓ϗετͷϦιʔε༗ݶ => ฒྻͷ੍ޚ ✓
͕͔͔࣌ؒΔॲཧ͕͋Δͱ࣍ͷδϣϒ͕࣮ߦ͞Εͳ͍ => λΠϜΞτͷઃఆ 31/43
ฒྻͷ੍ޚ ✓channel Λར༻ var sem = make(chan struct{}, 2) //
2ͭΛ༻ҙ func execute() { sem <- struct{}{} // ΛҰͭ֬อ͢Δ(ۭ͖͕ͳ͚Ε͜͜ͰॲཧΛͭ worker.execute() // CIδϣϒͷ࣮ߦ <-sem // ΛҰͭղ์͢Δ } func main() { go execute() go execute() go execute() } 32/43
λΠϜΞτͷઃఆ ✓context#WithTimeout ͰλΠϜΞτΛઃఆͰ͖Δ func execute() { err := make(chan error)
timeout, cancel := context.WithTimeout(context.Background(), 10 * time.Second) defer cancel() go func() { err <- worker.execute() // CIδϣϒͷ࣮ߦ }() select { case <-timeout.Done(): println("timeout") case e := <-err: println("done") } } 33/43
࣮ߦڥ (·ͱΊ) ✓DockerΫϥΠΞϯτ ✓ github.com/docker/docker ✓ ײతͳAPI ✓ ಉظॲཧ image
build ͱ container start ͰҟͳΔ ✓ฒྻͷ੍ޚ ✓ channel ✓λΠϜΞτͷઃఆ ✓ context#WithTimeout 34/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
35/43
࣮ߦ݁Ռ ✓GitHub ͷίϛοτεςʔλε 36/43
ίϛοτεςʔλε ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "ΞΫηετʔΫϯ"}) tc := oauth2.NewClient(context.Background(), ts) cli
:= github.NewClient(tc) stat := &github.Status{ Context: github.String("ίϛοτεςʔλεͷ໊લ"), Description: github.String("ৄࡉઆ໌ͳͲ(จࣈ੍ݶ͋Γ)"), State: github.String("εςʔλεʢྫ. failureʣ"), TargetURL: github.String("֎෦ϖʔδͷϦϯΫ"), } cli.Repositories.CreateStatus( context.Background(), "ownerName", "repoName", "ref (commit hash)", stat, ) 37/43
ϩά / Ϩεϙϯεͷදࣔ ✓http.ResponseWriter#Write ͰϨεϙϯεʹॻ͖ࠐΈ func LogHandler(w http.ResponseWriter, r *http.Request)
{ w.WriteHeader(http.StatusOK) w.Write([]byte("log")) } 38/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ૄͳઃܭʹ͓ͯ͘͠ͱࠩͰػೳΛ࣮͍͢͠ ✓݁ՌΛग़ྗ͍ͨ͠λΠϛϯάδϣϒ࣮ߦத(લޙ)ʹ ✓ δϣϒͷొ࣌ ✓ δϣϒͷ࣮ߦ։࢝࣌ ✓ ϩά ✓
δϣϒͷऴྃ࣌ʢޭ/ࣦഊ/Τϥʔ) 39/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓࣮ߦڥͰϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ type worker struct { startFun func() } func
(w *worker) run() { err := w.startFun() ... } func localRun() { w := &worker{ startFun: printLog } w.run() } func serverRun() { w := &worker{ startFun: printLogAndCommitStatus } w.run() } 40/43
࣮ߦ݁Ռ (·ͱΊ) ✓GitHubͷίϛοτεςʔλε ✓ github.com/google/go-github ✓Ϩεϙϯε ✓ net/http ͷ http.ResponseWriter#Write
✓࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ ϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ 41/43
·ͱΊ ✓ඪ४ػೳ/ඪ४ϥΠϒϥϦ ✓ ඇಉظλΠϜΞτ ✓ HTTPαʔόʔ ✓GitHub Docker ͳͲͷΫϥΠΞϯτϥΠϒϥϦΛར༻
✓ҰͭҰͭͷʢهड़ʣ؆ܿ ✓ Έ߹ΘͤͯCIΛ࡞Δ͜ͱ͕Ͱ͖Δ 42/43
Let's make CI server! 43/43