Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
GolangでDockerベースのCIを作る
Search
Shunsuke Maeda
May 18, 2019
Technology
3
3.7k
GolangでDockerベースのCIを作る
Golang を使って Dockerベース のCIを作るお話です。
Shunsuke Maeda
May 18, 2019
Tweet
Share
More Decks by Shunsuke Maeda
See All by Shunsuke Maeda
静的解析ツール detekt で任意の条件で警告させる
duck8823
1
1.4k
GitHub と連携する CI を作る
duck8823
3
2.6k
Other Decks in Technology
See All in Technology
5min GuardDuty Extended Threat Detection EKS
takakuni
0
190
タイミーのデータモデリング事例と今後のチャレンジ
ttccddtoki
6
2.3k
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
soudai
PRO
6
960
開発生産性を組織全体の「生産性」へ! 部門間連携の壁を越える実践的ステップ
sudo5in5k
2
5.9k
AIとともに進化するエンジニアリング / Engineering-Evolving-with-AI_final.pdf
lycorptech_jp
PRO
0
150
Tokyo_reInforce_2025_recap_iam_access_analyzer
hiashisan
0
170
Geminiとv0による高速プロトタイピング
shinya337
0
250
Beyond Kaniko: Navigating Unprivileged Container Image Creation
f30
0
130
「クラウドコスト絶対削減」を支える技術—FinOpsを超えた徹底的なクラウドコスト削減の実践論
delta_tech
4
140
MobileActOsaka_250704.pdf
akaitadaaki
0
110
Lambda Web Adapterについて自分なりに理解してみた
smt7174
6
160
生まれ変わった AWS Security Hub (Preview) を紹介 #reInforce_osaka / reInforce New Security Hub
masahirokawahara
0
430
Featured
See All Featured
Statistics for Hackers
jakevdp
799
220k
Testing 201, or: Great Expectations
jmmastey
42
7.6k
How to train your dragon (web standard)
notwaldorf
94
6.1k
Producing Creativity
orderedlist
PRO
346
40k
A designer walks into a library…
pauljervisheath
207
24k
Optimising Largest Contentful Paint
csswizardry
37
3.3k
Site-Speed That Sticks
csswizardry
10
680
Build The Right Thing And Hit Your Dates
maggiecrowley
36
2.8k
GraphQLとの向き合い方2022年版
quramy
49
14k
Docker and Python
trallard
44
3.5k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
281
13k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.9k
Transcript
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ ɹ GoConference 2019 Spring
2019.05.18 [SAT] Shunsuke Maeda (@duck8823)
ࣗݾհ ✓ Shunsuke Maeda ✓ @duck8823 ✓ ॴଐ ✓גࣜձࣾΤεɾΤϜɾΤε ✓EarthCampusגࣜձࣾʢ෭ʣ
2/43
CI͠ΜͲ͍ 3/43
CI͠ΜͲ͍ ! ✓CIΛߏங͍ͨ͠ ci := ci.New() for ci.IsBroken { //
CI͕յΕͯͨΒ fix(ci) // ϩʔΧϧͰ͚͢Ͳಈ࡞֬ೝͰ͖ͳ͍ͷͰ git.Commit() // ίϛοτ ͯ͠ git.Push() // ϓογϡ ͯ͠ ci.Run() // CI ಈ͔ͯ͠ΈΔ } 4/43
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ duck8823/duci 5/43
duci ͷಛ ✓DockerfileͰఆٛ ✓ ֶशίετ͕͍ ✓ ϩʔΧϧͰ࣮ߦ ✓GitHub ʹରԠ ✓
Push / PR্ͷίϝϯτ Ͱ࣮ߦ ✓ ίϛοτεςʔλεΛ࡞ 6/43
ֶशίετ ✓Dockerfile ͷΈ FROM golang:1.12.4-alpine # ϕʔεΠϝʔδ RUN apk --update
add --no-cache ... # ඞཁͳύοέʔδͷΠϯετʔϧ WORKDIR /workdir COPY . . ENTRYPOINT ["make"] # λεΫϥϯφʔͷར༻Λਪ CMD ["test"] # σϑΥϧτͷλεΫ 7/43
ֶशίετ ✓DockerͷࣝͰΩϟογϡԽ FROM golang:1.12.4-alpine RUN apk --update add --no-cache ...
WORKDIR /workdir COPY go.mod . COPY go.sum . RUN go mod download COPY . . ENTRYPOINT ["make"] CMD ["test"] 8/43
ϩʔΧϧͰ࠶ݱ ✓ίϚϯυΛ༻ҙ ✓ϩʔΧϧͰࢼ͔ͯ͠Β commit & push Ͱ͖Δ duci run INFO[14/May/2019
09:25:57.353] Step 1/9 : FROM openjdk:11 as Build INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> 0aa10063a184 INFO[14/May/2019 09:25:57.353] Step 2/9 : WORKDIR /workdir INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> Using cache INFO[14/May/2019 09:25:57.353] ---> 2dcc968d8994 ... 9/43
CIΛ࡞Δ 10/43
Continuous Integration 11/43
Continuous Integration ✓ఆظతʹ ✓ϏϧυςετΛ࣮ߦ ͢Δ͜ͱͰ ✓ૣظʹ ϑΟʔυόοΫ Λಘͯ όά͕ຊ൪ڥʹࠞೖ͢ΔͷΛ͙ 12/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
13/43
duci ʹ͓͚Δߏ ✓࣮ߦλΠϛϯά ✓ GitHub Webhooks ✓࣮ߦڥ ✓ Dockerίϯςφ ✓࣮ߦ݁Ռ
✓ ϩάग़ྗ / GitHub ίϛοτεςʔλε 14/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
15/43
࣮ߦλΠϛϯά ✓GitHub Webhooks https://github.com/<owner>/<repo>/settings/hooks ✓ ҙͷλΠϛϯάͰ Payload Λඈ͢ 16/43
PayloadΛड͚औΔ ✓GitHub ͔ΒඈΜͰ͘Δ Payload JSON Λड͚औΔඞཁ͕͋Δ import "net/http" func JobHandler(w
http.ResponseWriter, r *http.Request) { // ֤ΠϕϯτʹରԠͨ͠PayloadͷύʔεॲཧͱCIδϣϒͷ࣮ߦ } func main() { http.HandleFunc("/", JobHandler) http.ListenAndServe(":8080", nil) } 17/43
GitHub ͷ Webhooks ✓Event ͱ Action ͷΈ߹ΘͤͰλΠϛϯά੍ޚ ✓Event ✓ Webhooksͷछྨ
(Header: X-GitHub-Event) ྫ. push, issue_comment, pull_request ✓ GitHubͰઃఆՄೳ ✓Action ✓ Eventຖͷࡉ͔͍छྨ (JSON payload) ྫ. created, opened, synchronize 18/43
Eventͷऔಘ ✓ϦΫΤετͷϔομʔ X-GitHub-Event ͔ΒΠϕϯτΛऔಘ ✓Πϕϯτຖʹ Payload ͕ҟͳΔ func JobHandler(w http.ResponseWriter,
r *http.Request) { event := r.Header.Get("X-GitHub-Event") switch event { case "push": // https://developer.github.com/v3/activity/events/types/#pushevent case "issue_comment": // https://developer.github.com/v3/activity/events/types/#issuecommentevent case "pull_request": // https://developer.github.com/v3/activity/events/types/#pullrequestevent default: http.Error(w, fmt.Sprintf("Bad event type: %s", event), http.StatusBadRequest) } } 19/43
Payload JSON ͷऔΓѻ͍ ✓encoding/json ͰσίʔυͰ͖Δ ✓Payloadͷܕ github.com/google/go-github import ( "encoding/json"
"github.com/google/go-github/github" ) func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "push": event := &github.PushEvent{} err := json.NewDecoder(r.Body).Decode(event) ... } } 20/43
Forkͨ͠ϦϙδτϦ͔Βͷ Pull Request ✓push ΠϕϯτඈΜͰ͜ͳ͍ ✓ pull_request Πϕϯτͷ synchronize ΞΫγϣϯ
func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "pull_request": event := &github.PullRequestEvent{} json.NewDecoder(r.Body).Decode(event) switch event.GetAction() { case "synchronize", "open": ... } } } 21/43
Pull Request ͷίϝϯτ import ( "github.com/google/go-github/github" "golang.org/x/oauth2" ) func JobHandler(w
http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "issue_comment": event := &github.IssueComment{} json.NewDecoder(r.Body).Decode(event) ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "access token"}) tc := oauth2.NewClient(context.Background(), ts) cli := github.NewClient(tc) pr := cli.PullRequests.Get( context.Background(), ownerName, repoName, event.GetIssue().GetNumber(), ) ... } } 22/43
ඇಉظͰϨεϙϯεΛฦ͢ ✓δϣϒ͕͔͔࣌ؒΔ => ඇಉظʹ࣮ߦͯ͠ϨεϙϯεΛฦ͢ func JobHandler(w http.ResponseWriter, r *http.Request) {
if (invalidRequest(r)) { http.Error(w, r.Error(), http.StatusBadRequest) return } go func() { // CIδϣϒͷ࣮ߦ }() w.WriteHeader(http.StatusOK) } 23/43
࣮ߦλΠϛϯά (·ͱΊ) ✓ϦΫΤετड => net/http ✓Payload ͷܕ => ϥΠϒϥϦͷར༻ ྫ.
GitHub Webhooks: github.com/google/go-github ✓ GitHub Event ͱ Action ͰλΠϛϯάΛ੍ޚ ✓ඇಉظͰϨεϙϯεΛฦ͢ => goroutine 24/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
25/43
࣮ߦڥ ✓Dockerίϯςφ ✓ ϗετΛԚ͞ͳ͍ ✓ ϩʔΧϧ / CI Ͱͷಈ࡞ʹ࠶ݱੑΛͱΓ͍͢ ✓github.com/docker/docker
Λར༻ 26/43
github.com/docker/docker ✓ϦϙδτϦ moby/moby ✓Docker Daemon ͷରԠAPIόʔδϣϯ ɹ- Docker for Mac
1.39 ✓ Moby ͷ master 1.41 docker version Server: Docker Engine - Community Engine: Version: 18.09.2 API version: 1.39 (minimum version 1.12) ... 27/43
DockerΠϝʔδ/ίϯςφΛѻ͏ import "github.com/docker/docker/client" cli, _ := client.NewClientWithOpts(client.FromEnv) ✓ײతͳAPI ✓ cli#ImageBuild
= docker build ✓ cli#ContainerCreate = docker create ✓ cli#ContainerStart = docker start 28/43
DockerΠϝʔδͷϏϧυ ✓ίϯςΩετʢσΟϨΫτϦʣΛ tarball ͷܗࣜʹ͢Δඞཁ͕͋Δ ✓cli#ImageBuild ✓ ඇಉظʹ࣮ߦ͞ΕΔ ✓ resp.Body Λ
EOF ·ͰಡΜͰऴྃͪ opts := types.ImageBuildOptions{ Tags: []string{"tag"} } resp, _ := cli.ImageBuild(ctx, tarball("/workdir"), opts) ioutil.ReadAll(resp.Body) 29/43
ίϯςφͷ࣮ߦ ✓ίϯςφͷ࡞ cli#ContainerCreate ✓cli#ContainerStart ඇಉظ ✓ cli#ContainerWait cli#ContainerLogs ͰδϣϒͷใΛऔಘ
sopts := types.ContainerStartOptions{} cli.ContainerStart(context.Background(), "containerId", sopts) code, _ := cli.ContainerWait(context.Background(), "containerId") if code != 0 { // exit code ͕ 0Ҏ֎ => δϣϒͷࣦഊ } lopts := types.ContainerLogsOptions{ ShowStdout: true, ShowStderr: true } log _ := cli.ContainerLogs(context.Background(), "containerId", lopts) 30/43
δϣϒͷฒྻͱλΠϜΞτ ✓ϦΫΤετຖʹ goroutine Ͱ࣮ߦ ✓ δϣϒΛ͍࣮ͭ͘ߦͯ͠͠·͏ ✓ϗετͷϦιʔε༗ݶ => ฒྻͷ੍ޚ ✓
͕͔͔࣌ؒΔॲཧ͕͋Δͱ࣍ͷδϣϒ͕࣮ߦ͞Εͳ͍ => λΠϜΞτͷઃఆ 31/43
ฒྻͷ੍ޚ ✓channel Λར༻ var sem = make(chan struct{}, 2) //
2ͭΛ༻ҙ func execute() { sem <- struct{}{} // ΛҰͭ֬อ͢Δ(ۭ͖͕ͳ͚Ε͜͜ͰॲཧΛͭ worker.execute() // CIδϣϒͷ࣮ߦ <-sem // ΛҰͭղ์͢Δ } func main() { go execute() go execute() go execute() } 32/43
λΠϜΞτͷઃఆ ✓context#WithTimeout ͰλΠϜΞτΛઃఆͰ͖Δ func execute() { err := make(chan error)
timeout, cancel := context.WithTimeout(context.Background(), 10 * time.Second) defer cancel() go func() { err <- worker.execute() // CIδϣϒͷ࣮ߦ }() select { case <-timeout.Done(): println("timeout") case e := <-err: println("done") } } 33/43
࣮ߦڥ (·ͱΊ) ✓DockerΫϥΠΞϯτ ✓ github.com/docker/docker ✓ ײతͳAPI ✓ ಉظॲཧ image
build ͱ container start ͰҟͳΔ ✓ฒྻͷ੍ޚ ✓ channel ✓λΠϜΞτͷઃఆ ✓ context#WithTimeout 34/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
35/43
࣮ߦ݁Ռ ✓GitHub ͷίϛοτεςʔλε 36/43
ίϛοτεςʔλε ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "ΞΫηετʔΫϯ"}) tc := oauth2.NewClient(context.Background(), ts) cli
:= github.NewClient(tc) stat := &github.Status{ Context: github.String("ίϛοτεςʔλεͷ໊લ"), Description: github.String("ৄࡉઆ໌ͳͲ(จࣈ੍ݶ͋Γ)"), State: github.String("εςʔλεʢྫ. failureʣ"), TargetURL: github.String("֎෦ϖʔδͷϦϯΫ"), } cli.Repositories.CreateStatus( context.Background(), "ownerName", "repoName", "ref (commit hash)", stat, ) 37/43
ϩά / Ϩεϙϯεͷදࣔ ✓http.ResponseWriter#Write ͰϨεϙϯεʹॻ͖ࠐΈ func LogHandler(w http.ResponseWriter, r *http.Request)
{ w.WriteHeader(http.StatusOK) w.Write([]byte("log")) } 38/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ૄͳઃܭʹ͓ͯ͘͠ͱࠩͰػೳΛ࣮͍͢͠ ✓݁ՌΛग़ྗ͍ͨ͠λΠϛϯάδϣϒ࣮ߦத(લޙ)ʹ ✓ δϣϒͷొ࣌ ✓ δϣϒͷ࣮ߦ։࢝࣌ ✓ ϩά ✓
δϣϒͷऴྃ࣌ʢޭ/ࣦഊ/Τϥʔ) 39/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓࣮ߦڥͰϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ type worker struct { startFun func() } func
(w *worker) run() { err := w.startFun() ... } func localRun() { w := &worker{ startFun: printLog } w.run() } func serverRun() { w := &worker{ startFun: printLogAndCommitStatus } w.run() } 40/43
࣮ߦ݁Ռ (·ͱΊ) ✓GitHubͷίϛοτεςʔλε ✓ github.com/google/go-github ✓Ϩεϙϯε ✓ net/http ͷ http.ResponseWriter#Write
✓࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ ϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ 41/43
·ͱΊ ✓ඪ४ػೳ/ඪ४ϥΠϒϥϦ ✓ ඇಉظλΠϜΞτ ✓ HTTPαʔόʔ ✓GitHub Docker ͳͲͷΫϥΠΞϯτϥΠϒϥϦΛར༻
✓ҰͭҰͭͷʢهड़ʣ؆ܿ ✓ Έ߹ΘͤͯCIΛ࡞Δ͜ͱ͕Ͱ͖Δ 42/43
Let's make CI server! 43/43