Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
GolangでDockerベースのCIを作る
Search
Shunsuke Maeda
May 18, 2019
Technology
3
3.4k
GolangでDockerベースのCIを作る
Golang を使って Dockerベース のCIを作るお話です。
Shunsuke Maeda
May 18, 2019
Tweet
Share
More Decks by Shunsuke Maeda
See All by Shunsuke Maeda
静的解析ツール detekt で任意の条件で警告させる
duck8823
1
1.3k
GitHub と連携する CI を作る
duck8823
3
2.4k
Other Decks in Technology
See All in Technology
LeSSに潜む「隠れWF病」とその処方箋
lycorptech_jp
PRO
2
120
顧客が本当に必要だったもの - パフォーマンス改善編 / Make what is needed
soudai
24
6.8k
Autify Company Deck
autifyhq
1
39k
「視座」の上げ方が成人発達理論にわかりやすくまとまってた / think_ perspective_hidden_dimensions
shuzon
2
4.1k
「 SharePoint 難しい」ってよく聞くけど、そんなに言うなら8歳の息子に試してもらった
taichinakamura
1
620
Aurora_BlueGreenDeploymentsやってみた
tsukasa_ishimaru
1
120
신뢰할 수 있는 AI 검색 엔진을 만들기 위한 Liner의 여정
huffon
0
350
グローバル展開を見据えたサービスにおける機械翻訳プラクティス / dp-ai-translating
cyberagentdevelopers
PRO
1
150
Product Engineer Night #6プロダクトエンジニアを育む仕組み・施策
hacomono
PRO
1
470
来年もre:Invent2024 に行きたいあなたへ - “集中”と“つながり”で楽しむ -
ny7760
0
470
AWS CodePipelineでコンテナアプリをデプロイした際に、古いイメージを自動で削除する
smt7174
1
100
フルカイテン株式会社 採用資料
fullkaiten
0
36k
Featured
See All Featured
Ruby is Unlike a Banana
tanoku
96
11k
RailsConf 2023
tenderlove
29
880
GraphQLとの向き合い方2022年版
quramy
43
13k
Building an army of robots
kneath
302
42k
Why You Should Never Use an ORM
jnunemaker
PRO
53
9k
Facilitating Awesome Meetings
lara
49
6k
Typedesign – Prime Four
hannesfritz
39
2.4k
KATA
mclloyd
29
13k
Scaling GitHub
holman
458
140k
The Power of CSS Pseudo Elements
geoffreycrofte
72
5.3k
The Language of Interfaces
destraynor
154
24k
StorybookのUI Testing Handbookを読んだ
zakiyama
26
5.2k
Transcript
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ ɹ GoConference 2019 Spring
2019.05.18 [SAT] Shunsuke Maeda (@duck8823)
ࣗݾհ ✓ Shunsuke Maeda ✓ @duck8823 ✓ ॴଐ ✓גࣜձࣾΤεɾΤϜɾΤε ✓EarthCampusגࣜձࣾʢ෭ʣ
2/43
CI͠ΜͲ͍ 3/43
CI͠ΜͲ͍ ! ✓CIΛߏங͍ͨ͠ ci := ci.New() for ci.IsBroken { //
CI͕յΕͯͨΒ fix(ci) // ϩʔΧϧͰ͚͢Ͳಈ࡞֬ೝͰ͖ͳ͍ͷͰ git.Commit() // ίϛοτ ͯ͠ git.Push() // ϓογϡ ͯ͠ ci.Run() // CI ಈ͔ͯ͠ΈΔ } 4/43
Golang Ͱ Docker ϕʔεͷ CI Λ࡞Δ duck8823/duci 5/43
duci ͷಛ ✓DockerfileͰఆٛ ✓ ֶशίετ͕͍ ✓ ϩʔΧϧͰ࣮ߦ ✓GitHub ʹରԠ ✓
Push / PR্ͷίϝϯτ Ͱ࣮ߦ ✓ ίϛοτεςʔλεΛ࡞ 6/43
ֶशίετ ✓Dockerfile ͷΈ FROM golang:1.12.4-alpine # ϕʔεΠϝʔδ RUN apk --update
add --no-cache ... # ඞཁͳύοέʔδͷΠϯετʔϧ WORKDIR /workdir COPY . . ENTRYPOINT ["make"] # λεΫϥϯφʔͷར༻Λਪ CMD ["test"] # σϑΥϧτͷλεΫ 7/43
ֶशίετ ✓DockerͷࣝͰΩϟογϡԽ FROM golang:1.12.4-alpine RUN apk --update add --no-cache ...
WORKDIR /workdir COPY go.mod . COPY go.sum . RUN go mod download COPY . . ENTRYPOINT ["make"] CMD ["test"] 8/43
ϩʔΧϧͰ࠶ݱ ✓ίϚϯυΛ༻ҙ ✓ϩʔΧϧͰࢼ͔ͯ͠Β commit & push Ͱ͖Δ duci run INFO[14/May/2019
09:25:57.353] Step 1/9 : FROM openjdk:11 as Build INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> 0aa10063a184 INFO[14/May/2019 09:25:57.353] Step 2/9 : WORKDIR /workdir INFO[14/May/2019 09:25:57.353] INFO[14/May/2019 09:25:57.353] ---> Using cache INFO[14/May/2019 09:25:57.353] ---> 2dcc968d8994 ... 9/43
CIΛ࡞Δ 10/43
Continuous Integration 11/43
Continuous Integration ✓ఆظతʹ ✓ϏϧυςετΛ࣮ߦ ͢Δ͜ͱͰ ✓ૣظʹ ϑΟʔυόοΫ Λಘͯ όά͕ຊ൪ڥʹࠞೖ͢ΔͷΛ͙ 12/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
13/43
duci ʹ͓͚Δߏ ✓࣮ߦλΠϛϯά ✓ GitHub Webhooks ✓࣮ߦڥ ✓ Dockerίϯςφ ✓࣮ߦ݁Ռ
✓ ϩάग़ྗ / GitHub ίϛοτεςʔλε 14/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
15/43
࣮ߦλΠϛϯά ✓GitHub Webhooks https://github.com/<owner>/<repo>/settings/hooks ✓ ҙͷλΠϛϯάͰ Payload Λඈ͢ 16/43
PayloadΛड͚औΔ ✓GitHub ͔ΒඈΜͰ͘Δ Payload JSON Λड͚औΔඞཁ͕͋Δ import "net/http" func JobHandler(w
http.ResponseWriter, r *http.Request) { // ֤ΠϕϯτʹରԠͨ͠PayloadͷύʔεॲཧͱCIδϣϒͷ࣮ߦ } func main() { http.HandleFunc("/", JobHandler) http.ListenAndServe(":8080", nil) } 17/43
GitHub ͷ Webhooks ✓Event ͱ Action ͷΈ߹ΘͤͰλΠϛϯά੍ޚ ✓Event ✓ Webhooksͷछྨ
(Header: X-GitHub-Event) ྫ. push, issue_comment, pull_request ✓ GitHubͰઃఆՄೳ ✓Action ✓ Eventຖͷࡉ͔͍छྨ (JSON payload) ྫ. created, opened, synchronize 18/43
Eventͷऔಘ ✓ϦΫΤετͷϔομʔ X-GitHub-Event ͔ΒΠϕϯτΛऔಘ ✓Πϕϯτຖʹ Payload ͕ҟͳΔ func JobHandler(w http.ResponseWriter,
r *http.Request) { event := r.Header.Get("X-GitHub-Event") switch event { case "push": // https://developer.github.com/v3/activity/events/types/#pushevent case "issue_comment": // https://developer.github.com/v3/activity/events/types/#issuecommentevent case "pull_request": // https://developer.github.com/v3/activity/events/types/#pullrequestevent default: http.Error(w, fmt.Sprintf("Bad event type: %s", event), http.StatusBadRequest) } } 19/43
Payload JSON ͷऔΓѻ͍ ✓encoding/json ͰσίʔυͰ͖Δ ✓Payloadͷܕ github.com/google/go-github import ( "encoding/json"
"github.com/google/go-github/github" ) func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "push": event := &github.PushEvent{} err := json.NewDecoder(r.Body).Decode(event) ... } } 20/43
Forkͨ͠ϦϙδτϦ͔Βͷ Pull Request ✓push ΠϕϯτඈΜͰ͜ͳ͍ ✓ pull_request Πϕϯτͷ synchronize ΞΫγϣϯ
func JobHandler(w http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "pull_request": event := &github.PullRequestEvent{} json.NewDecoder(r.Body).Decode(event) switch event.GetAction() { case "synchronize", "open": ... } } } 21/43
Pull Request ͷίϝϯτ import ( "github.com/google/go-github/github" "golang.org/x/oauth2" ) func JobHandler(w
http.ResponseWriter, r *http.Request) { switch r.Header.Get("X-GitHub-Event") { case "issue_comment": event := &github.IssueComment{} json.NewDecoder(r.Body).Decode(event) ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "access token"}) tc := oauth2.NewClient(context.Background(), ts) cli := github.NewClient(tc) pr := cli.PullRequests.Get( context.Background(), ownerName, repoName, event.GetIssue().GetNumber(), ) ... } } 22/43
ඇಉظͰϨεϙϯεΛฦ͢ ✓δϣϒ͕͔͔࣌ؒΔ => ඇಉظʹ࣮ߦͯ͠ϨεϙϯεΛฦ͢ func JobHandler(w http.ResponseWriter, r *http.Request) {
if (invalidRequest(r)) { http.Error(w, r.Error(), http.StatusBadRequest) return } go func() { // CIδϣϒͷ࣮ߦ }() w.WriteHeader(http.StatusOK) } 23/43
࣮ߦλΠϛϯά (·ͱΊ) ✓ϦΫΤετड => net/http ✓Payload ͷܕ => ϥΠϒϥϦͷར༻ ྫ.
GitHub Webhooks: github.com/google/go-github ✓ GitHub Event ͱ Action ͰλΠϛϯάΛ੍ޚ ✓ඇಉظͰϨεϙϯεΛฦ͢ => goroutine 24/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
25/43
࣮ߦڥ ✓Dockerίϯςφ ✓ ϗετΛԚ͞ͳ͍ ✓ ϩʔΧϧ / CI Ͱͷಈ࡞ʹ࠶ݱੑΛͱΓ͍͢ ✓github.com/docker/docker
Λར༻ 26/43
github.com/docker/docker ✓ϦϙδτϦ moby/moby ✓Docker Daemon ͷରԠAPIόʔδϣϯ ɹ- Docker for Mac
1.39 ✓ Moby ͷ master 1.41 docker version Server: Docker Engine - Community Engine: Version: 18.09.2 API version: 1.39 (minimum version 1.12) ... 27/43
DockerΠϝʔδ/ίϯςφΛѻ͏ import "github.com/docker/docker/client" cli, _ := client.NewClientWithOpts(client.FromEnv) ✓ײతͳAPI ✓ cli#ImageBuild
= docker build ✓ cli#ContainerCreate = docker create ✓ cli#ContainerStart = docker start 28/43
DockerΠϝʔδͷϏϧυ ✓ίϯςΩετʢσΟϨΫτϦʣΛ tarball ͷܗࣜʹ͢Δඞཁ͕͋Δ ✓cli#ImageBuild ✓ ඇಉظʹ࣮ߦ͞ΕΔ ✓ resp.Body Λ
EOF ·ͰಡΜͰऴྃͪ opts := types.ImageBuildOptions{ Tags: []string{"tag"} } resp, _ := cli.ImageBuild(ctx, tarball("/workdir"), opts) ioutil.ReadAll(resp.Body) 29/43
ίϯςφͷ࣮ߦ ✓ίϯςφͷ࡞ cli#ContainerCreate ✓cli#ContainerStart ඇಉظ ✓ cli#ContainerWait cli#ContainerLogs ͰδϣϒͷใΛऔಘ
sopts := types.ContainerStartOptions{} cli.ContainerStart(context.Background(), "containerId", sopts) code, _ := cli.ContainerWait(context.Background(), "containerId") if code != 0 { // exit code ͕ 0Ҏ֎ => δϣϒͷࣦഊ } lopts := types.ContainerLogsOptions{ ShowStdout: true, ShowStderr: true } log _ := cli.ContainerLogs(context.Background(), "containerId", lopts) 30/43
δϣϒͷฒྻͱλΠϜΞτ ✓ϦΫΤετຖʹ goroutine Ͱ࣮ߦ ✓ δϣϒΛ͍࣮ͭ͘ߦͯ͠͠·͏ ✓ϗετͷϦιʔε༗ݶ => ฒྻͷ੍ޚ ✓
͕͔͔࣌ؒΔॲཧ͕͋Δͱ࣍ͷδϣϒ͕࣮ߦ͞Εͳ͍ => λΠϜΞτͷઃఆ 31/43
ฒྻͷ੍ޚ ✓channel Λར༻ var sem = make(chan struct{}, 2) //
2ͭΛ༻ҙ func execute() { sem <- struct{}{} // ΛҰͭ֬อ͢Δ(ۭ͖͕ͳ͚Ε͜͜ͰॲཧΛͭ worker.execute() // CIδϣϒͷ࣮ߦ <-sem // ΛҰͭղ์͢Δ } func main() { go execute() go execute() go execute() } 32/43
λΠϜΞτͷઃఆ ✓context#WithTimeout ͰλΠϜΞτΛઃఆͰ͖Δ func execute() { err := make(chan error)
timeout, cancel := context.WithTimeout(context.Background(), 10 * time.Second) defer cancel() go func() { err <- worker.execute() // CIδϣϒͷ࣮ߦ }() select { case <-timeout.Done(): println("timeout") case e := <-err: println("done") } } 33/43
࣮ߦڥ (·ͱΊ) ✓DockerΫϥΠΞϯτ ✓ github.com/docker/docker ✓ ײతͳAPI ✓ ಉظॲཧ image
build ͱ container start ͰҟͳΔ ✓ฒྻͷ੍ޚ ✓ channel ✓λΠϜΞτͷઃఆ ✓ context#WithTimeout 34/43
CIͰߟ͑Δඞཁ͕͋Δ͜ͱ ✓ఆظతʹ ✓ ࣮ߦλΠϛϯά ✓ϏϧυςετΛ࣮ߦ ✓ ࣮ߦڥ ✓ϑΟʔυόοΫ ✓ ࣮ߦ݁Ռ
35/43
࣮ߦ݁Ռ ✓GitHub ͷίϛοτεςʔλε 36/43
ίϛοτεςʔλε ts := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: "ΞΫηετʔΫϯ"}) tc := oauth2.NewClient(context.Background(), ts) cli
:= github.NewClient(tc) stat := &github.Status{ Context: github.String("ίϛοτεςʔλεͷ໊લ"), Description: github.String("ৄࡉઆ໌ͳͲ(จࣈ੍ݶ͋Γ)"), State: github.String("εςʔλεʢྫ. failureʣ"), TargetURL: github.String("֎෦ϖʔδͷϦϯΫ"), } cli.Repositories.CreateStatus( context.Background(), "ownerName", "repoName", "ref (commit hash)", stat, ) 37/43
ϩά / Ϩεϙϯεͷදࣔ ✓http.ResponseWriter#Write ͰϨεϙϯεʹॻ͖ࠐΈ func LogHandler(w http.ResponseWriter, r *http.Request)
{ w.WriteHeader(http.StatusOK) w.Write([]byte("log")) } 38/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ૄͳઃܭʹ͓ͯ͘͠ͱࠩͰػೳΛ࣮͍͢͠ ✓݁ՌΛग़ྗ͍ͨ͠λΠϛϯάδϣϒ࣮ߦத(લޙ)ʹ ✓ δϣϒͷొ࣌ ✓ δϣϒͷ࣮ߦ։࢝࣌ ✓ ϩά ✓
δϣϒͷऴྃ࣌ʢޭ/ࣦഊ/Τϥʔ) 39/43
࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓࣮ߦڥͰϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ type worker struct { startFun func() } func
(w *worker) run() { err := w.startFun() ... } func localRun() { w := &worker{ startFun: printLog } w.run() } func serverRun() { w := &worker{ startFun: printLogAndCommitStatus } w.run() } 40/43
࣮ߦ݁Ռ (·ͱΊ) ✓GitHubͷίϛοτεςʔλε ✓ github.com/google/go-github ✓Ϩεϙϯε ✓ net/http ͷ http.ResponseWriter#Write
✓࣮ߦڥͱ࣮ߦ݁ՌΛૄʹ͢Δ ✓ ϑΟʔυόοΫ༻ؔΛઃఆͰ͖ΔΑ͏ʹ͢Δ 41/43
·ͱΊ ✓ඪ४ػೳ/ඪ४ϥΠϒϥϦ ✓ ඇಉظλΠϜΞτ ✓ HTTPαʔόʔ ✓GitHub Docker ͳͲͷΫϥΠΞϯτϥΠϒϥϦΛར༻
✓ҰͭҰͭͷʢهड़ʣ؆ܿ ✓ Έ߹ΘͤͯCIΛ࡞Δ͜ͱ͕Ͱ͖Δ 42/43
Let's make CI server! 43/43