Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
GolangでDockerベースのCIを作る
Search
Shunsuke Maeda
May 18, 2019
Technology
3
3.8k
GolangでDockerベースのCIを作る
Golang を使って Dockerベース のCIを作るお話です。
Shunsuke Maeda
May 18, 2019
Tweet
Share
More Decks by Shunsuke Maeda
See All by Shunsuke Maeda
静的解析ツール detekt で任意の条件で警告させる
duck8823
1
1.5k
GitHub と連携する CI を作る
duck8823
3
2.7k
Other Decks in Technology
See All in Technology
AI時代こそ求められる設計力- AWSクラウドデザインパターン3選で信頼性と拡張性を高める-
kenichirokimura
3
330
Wasmのエコシステムを使った ツール作成方法
askua
0
200
Claude Codeを駆使した初めてのiOSアプリ開発 ~ゼロから3週間でグローバルハッカソンで入賞するまで~
oikon48
10
4.7k
ガバメントクラウド(AWS)へのデータ移行戦略の立て方【虎の巻】 / 20251011 Mitsutosi Matsuo
shift_evolve
PRO
2
200
Introdução a Service Mesh usando o Istio
aeciopires
0
170
名刺メーカーDevグループ 紹介資料
sansan33
PRO
0
930
Sansan Engineering Unit 紹介資料
sansan33
PRO
1
3k
研究開発部メンバーの働き⽅ / Sansan R&D Profile
sansan33
PRO
3
20k
Railsの話をしよう
yahonda
0
160
スタートアップにおけるこれからの「データ整備」
shomaekawa
2
490
データ戦略部門 紹介資料
sansan33
PRO
1
3.8k
WEBサービスを成り立たせるAWSサービス
takano0131
1
180
Featured
See All Featured
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Building a Scalable Design System with Sketch
lauravandoore
463
33k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.5k
Learning to Love Humans: Emotional Interface Design
aarron
274
41k
A better future with KSS
kneath
239
18k
Git: the NoSQL Database
bkeepers
PRO
431
66k
A designer walks into a library…
pauljervisheath
209
24k
GraphQLの誤解/rethinking-graphql
sonatard
73
11k
Transcript
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ ɹ GoConference 2019 Spring
2019.05.18 [SAT] Shunsuke Maeda (@duck8823)
ࣗݾհ ✓ Shunsuke Maeda ✓ @duck8823 ✓ ॴଐ ✓גࣜձࣾΤεɾΤϜɾΤε ✓EarthCampusגࣜձࣾʢ෭ʣ
2/43
CI͠ΜͲ͍ 3/43
CI͠ΜͲ͍ ! ✓CIΛߏங͍ͨ͠ ci := ci.New() for ci.IsBroken { //
CI͕յΕͯͨΒ fix(ci) // ϩʔΧϧͰ͚͢Ͳಈ࡞֬ೝͰ͖ͳ͍ͷͰ git.Commit() // ίϛοτ ͯ͠ git.Push() // ϓογϡ ͯ͠ ci.Run() // CI ಈ͔ͯ͠ΈΔ } 4/43
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ duck8823/duci 5/43
duci ͷಛ ✓DockerfileͰఆٛ ✓ ֶशίετ͕͍ ✓ ϩʔΧϧͰ࣮ߦ ✓GitHub ʹରԠ ✓
Push / PR্ͷίϝϯτ Ͱ࣮ߦ ✓ ίϛοτεςʔλεΛ࡞ 6/43
ֶशίετ ✓Dockerfile ͷΈ FROM golang:1.12.4-alpine # ϕʔεΠϝʔδ RUN apk --update
add --no-cache ... # ඞཁͳύοέʔδͷΠϯετʔϧ WORKDIR /workdir COPY . . ENTRYPOINT ["make"] # λεΫϥϯφʔͷར༻Λਪ CMD ["test"] # σϑΥϧτͷλεΫ 7/43
ֶशίετ ✓DockerͷࣝͰΩϟογϡԽ FROM golang:1.12.4-alpine RUN apk --update add --no-cache ...
WORKDIR /workdir COPY go.mod . COPY go.sum . RUN go mod download COPY . . ENTRYPOINT ["make"] CMD ["test"] 8/43
ϩʔΧϧͰ࠶ݱ ✓ίϚϯυΛ༻ҙ ✓ϩʔΧϧͰࢼ͔ͯ͠Β commit & push Ͱ͖Δ duci run INFO[14/May/2019
09:25:57.353] Step 1/9 : FROM openjdk:11 as Build INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> 0aa10063a184 INFO[14/May/2019 09:25:57.353] Step 2/9 : WORKDIR /workdir INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> Using cache INFO[14/May/2019 09:25:57.353] ---> 2dcc968d8994 ... 9/43
CIΛ࡞Δ 10/43
Continuous Integration 11/43
Continuous Integration ✓ఆظతʹ ✓ϏϧυςετΛ࣮ߦ ͢Δ͜ͱͰ ✓ૣظʹ ϑΟʔυόοΫ Λಘͯ όά͕ຊ൪ڥʹࠞೖ͢ΔͷΛ͙ 12/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
13/43
duci ʹ͓͚Δߏ ✓࣮ߦλΠϛϯά ✓ GitHub Webhooks ✓࣮ߦڥ ✓ Dockerίϯςφ ✓࣮ߦ݁Ռ
✓ ϩάग़ྗ / GitHub ίϛοτεςʔλε 14/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
15/43
࣮ߦλΠϛϯά ✓GitHub Webhooks https://github.com/<owner>/<repo>/settings/hooks ✓ ҙͷλΠϛϯάͰ Payload Λඈ͢ 16/43
PayloadΛड͚औΔ ✓GitHub ͔ΒඈΜͰ͘Δ Payload JSON Λड͚औΔඞཁ͕͋Δ import "net/http" func JobHandler(w
http.ResponseWriter, r *http.Request) { // ֤ΠϕϯτʹରԠͨ͠PayloadͷύʔεॲཧͱCIδϣϒͷ࣮ߦ } func main() { http.HandleFunc("/", JobHandler) http.ListenAndServe(":8080", nil) } 17/43
GitHub ͷ Webhooks ✓Event ͱ Action ͷΈ߹ΘͤͰλΠϛϯά੍ޚ ✓Event ✓ Webhooksͷछྨ
(Header: X-GitHub-Event) ྫ. push, issue_comment, pull_request ✓ GitHubͰઃఆՄೳ ✓Action ✓ Eventຖͷࡉ͔͍छྨ (JSON payload) ྫ. created, opened, synchronize 18/43
Eventͷऔಘ ✓ϦΫΤετͷϔομʔ X-GitHub-Event ͔ΒΠϕϯτΛऔಘ ✓Πϕϯτຖʹ Payload ͕ҟͳΔ func JobHandler(w http.ResponseWriter,
r *http.Request) { event := r.Header.Get("X-GitHub-Event") switch event { case "push": // https://developer.github.com/v3/activity/events/types/#pushevent case "issue_comment": // https://developer.github.com/v3/activity/events/types/#issuecommentevent case "pull_request": // https://developer.github.com/v3/activity/events/types/#pullrequestevent default: http.Error(w, fmt.Sprintf("Bad event type: %s", event), http.StatusBadRequest) } } 19/43
Payload JSON ͷऔΓѻ͍ ✓encoding/json ͰσίʔυͰ͖Δ ✓Payloadͷܕ github.com/google/go-github import ( "encoding/json"
"github.com/google/go-github/github" ) func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "push": event := &github.PushEvent{} err := json.NewDecoder(r.Body).Decode(event) ... } } 20/43
Forkͨ͠ϦϙδτϦ͔Βͷ Pull Request ✓push ΠϕϯτඈΜͰ͜ͳ͍ ✓ pull_request Πϕϯτͷ synchronize ΞΫγϣϯ
func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "pull_request": event := &github.PullRequestEvent{} json.NewDecoder(r.Body).Decode(event) switch event.GetAction() { case "synchronize", "open": ... } } } 21/43
Pull Request ͷίϝϯτ import ( "github.com/google/go-github/github" "golang.org/x/oauth2" ) func JobHandler(w
http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "issue_comment": event := &github.IssueComment{} json.NewDecoder(r.Body).Decode(event) ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "access token"}) tc := oauth2.NewClient(context.Background(), ts) cli := github.NewClient(tc) pr := cli.PullRequests.Get( context.Background(), ownerName, repoName, event.GetIssue().GetNumber(), ) ... } } 22/43
ඇಉظͰϨεϙϯεΛฦ͢ ✓δϣϒ͕͔͔࣌ؒΔ => ඇಉظʹ࣮ߦͯ͠ϨεϙϯεΛฦ͢ func JobHandler(w http.ResponseWriter, r *http.Request) {
if (invalidRequest(r)) { http.Error(w, r.Error(), http.StatusBadRequest) return } go func() { // CIδϣϒͷ࣮ߦ }() w.WriteHeader(http.StatusOK) } 23/43
࣮ߦλΠϛϯά (·ͱΊ) ✓ϦΫΤετड => net/http ✓Payload ͷܕ => ϥΠϒϥϦͷར༻ ྫ.
GitHub Webhooks: github.com/google/go-github ✓ GitHub Event ͱ Action ͰλΠϛϯάΛ੍ޚ ✓ඇಉظͰϨεϙϯεΛฦ͢ => goroutine 24/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
25/43
࣮ߦڥ ✓Dockerίϯςφ ✓ ϗετΛԚ͞ͳ͍ ✓ ϩʔΧϧ / CI Ͱͷಈ࡞ʹ࠶ݱੑΛͱΓ͍͢ ✓github.com/docker/docker
Λར༻ 26/43
github.com/docker/docker ✓ϦϙδτϦ moby/moby ✓Docker Daemon ͷରԠAPIόʔδϣϯ ɹ- Docker for Mac
1.39 ✓ Moby ͷ master 1.41 docker version Server: Docker Engine - Community Engine: Version: 18.09.2 API version: 1.39 (minimum version 1.12) ... 27/43
DockerΠϝʔδ/ίϯςφΛѻ͏ import "github.com/docker/docker/client" cli, _ := client.NewClientWithOpts(client.FromEnv) ✓ײతͳAPI ✓ cli#ImageBuild
= docker build ✓ cli#ContainerCreate = docker create ✓ cli#ContainerStart = docker start 28/43
DockerΠϝʔδͷϏϧυ ✓ίϯςΩετʢσΟϨΫτϦʣΛ tarball ͷܗࣜʹ͢Δඞཁ͕͋Δ ✓cli#ImageBuild ✓ ඇಉظʹ࣮ߦ͞ΕΔ ✓ resp.Body Λ
EOF ·ͰಡΜͰऴྃͪ opts := types.ImageBuildOptions{ Tags: []string{"tag"} } resp, _ := cli.ImageBuild(ctx, tarball("/workdir"), opts) ioutil.ReadAll(resp.Body) 29/43
ίϯςφͷ࣮ߦ ✓ίϯςφͷ࡞ cli#ContainerCreate ✓cli#ContainerStart ඇಉظ ✓ cli#ContainerWait cli#ContainerLogs ͰδϣϒͷใΛऔಘ
sopts := types.ContainerStartOptions{} cli.ContainerStart(context.Background(), "containerId", sopts) code, _ := cli.ContainerWait(context.Background(), "containerId") if code != 0 { // exit code ͕ 0Ҏ֎ => δϣϒͷࣦഊ } lopts := types.ContainerLogsOptions{ ShowStdout: true, ShowStderr: true } log _ := cli.ContainerLogs(context.Background(), "containerId", lopts) 30/43
δϣϒͷฒྻͱλΠϜΞτ ✓ϦΫΤετຖʹ goroutine Ͱ࣮ߦ ✓ δϣϒΛ͍࣮ͭ͘ߦͯ͠͠·͏ ✓ϗετͷϦιʔε༗ݶ => ฒྻͷ੍ޚ ✓
͕͔͔࣌ؒΔॲཧ͕͋Δͱ࣍ͷδϣϒ͕࣮ߦ͞Εͳ͍ => λΠϜΞτͷઃఆ 31/43
ฒྻͷ੍ޚ ✓channel Λར༻ var sem = make(chan struct{}, 2) //
2ͭΛ༻ҙ func execute() { sem <- struct{}{} // ΛҰͭ֬อ͢Δ(ۭ͖͕ͳ͚Ε͜͜ͰॲཧΛͭ worker.execute() // CIδϣϒͷ࣮ߦ <-sem // ΛҰͭղ์͢Δ } func main() { go execute() go execute() go execute() } 32/43
λΠϜΞτͷઃఆ ✓context#WithTimeout ͰλΠϜΞτΛઃఆͰ͖Δ func execute() { err := make(chan error)
timeout, cancel := context.WithTimeout(context.Background(), 10 * time.Second) defer cancel() go func() { err <- worker.execute() // CIδϣϒͷ࣮ߦ }() select { case <-timeout.Done(): println("timeout") case e := <-err: println("done") } } 33/43
࣮ߦڥ (·ͱΊ) ✓DockerΫϥΠΞϯτ ✓ github.com/docker/docker ✓ ײతͳAPI ✓ ಉظॲཧ image
build ͱ container start ͰҟͳΔ ✓ฒྻͷ੍ޚ ✓ channel ✓λΠϜΞτͷઃఆ ✓ context#WithTimeout 34/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
35/43
࣮ߦ݁Ռ ✓GitHub ͷίϛοτεςʔλε 36/43
ίϛοτεςʔλε ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "ΞΫηετʔΫϯ"}) tc := oauth2.NewClient(context.Background(), ts) cli
:= github.NewClient(tc) stat := &github.Status{ Context: github.String("ίϛοτεςʔλεͷ໊લ"), Description: github.String("ৄࡉઆ໌ͳͲ(จࣈ੍ݶ͋Γ)"), State: github.String("εςʔλεʢྫ. failureʣ"), TargetURL: github.String("֎෦ϖʔδͷϦϯΫ"), } cli.Repositories.CreateStatus( context.Background(), "ownerName", "repoName", "ref (commit hash)", stat, ) 37/43
ϩά / Ϩεϙϯεͷදࣔ ✓http.ResponseWriter#Write ͰϨεϙϯεʹॻ͖ࠐΈ func LogHandler(w http.ResponseWriter, r *http.Request)
{ w.WriteHeader(http.StatusOK) w.Write([]byte("log")) } 38/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ૄͳઃܭʹ͓ͯ͘͠ͱࠩͰػೳΛ࣮͍͢͠ ✓݁ՌΛग़ྗ͍ͨ͠λΠϛϯάδϣϒ࣮ߦத(લޙ)ʹ ✓ δϣϒͷొ࣌ ✓ δϣϒͷ࣮ߦ։࢝࣌ ✓ ϩά ✓
δϣϒͷऴྃ࣌ʢޭ/ࣦഊ/Τϥʔ) 39/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓࣮ߦڥͰϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ type worker struct { startFun func() } func
(w *worker) run() { err := w.startFun() ... } func localRun() { w := &worker{ startFun: printLog } w.run() } func serverRun() { w := &worker{ startFun: printLogAndCommitStatus } w.run() } 40/43
࣮ߦ݁Ռ (·ͱΊ) ✓GitHubͷίϛοτεςʔλε ✓ github.com/google/go-github ✓Ϩεϙϯε ✓ net/http ͷ http.ResponseWriter#Write
✓࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ ϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ 41/43
·ͱΊ ✓ඪ४ػೳ/ඪ४ϥΠϒϥϦ ✓ ඇಉظλΠϜΞτ ✓ HTTPαʔόʔ ✓GitHub Docker ͳͲͷΫϥΠΞϯτϥΠϒϥϦΛར༻
✓ҰͭҰͭͷʢهड़ʣ؆ܿ ✓ Έ߹ΘͤͯCIΛ࡞Δ͜ͱ͕Ͱ͖Δ 42/43
Let's make CI server! 43/43