Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Applications for Container with Bazel

Building Applications for Container with Bazel

Avatar for Fumihiro Ito

Fumihiro Ito

October 10, 2020
Tweet

More Decks by Fumihiro Ito

Other Decks in Programming

Transcript

  1. ࣗݾ঺հ • Fumihiro Ito ʢҏ౻ ࢙ߒʣ • Software Engineer •

    ࠓ೥1݄ʹೖࣾ͠·ͨ͠ • θοτϥϘגࣜձࣾ • Ϡϑʔגࣜձࣾ100%ࢠձࣾ • ϠϑʔͷΠϯϑϥΛࢧ͑Δج൫ͷR&D • Kubernetes as a Service ͸ϠϑʔͰӡ༻͞Ε600ΫϥελҎ্Λ؅ཧ͍ͯ͠Δ 2
  2. Bazelͱ͸ • ΦʔϓϯιʔεͷϏϧυπʔϧ • GoogleͷΤϯδχΞ͕த৺ʹͳͬͯ։ൃΛ͍ͯ͠Δ • ίϛολʔͷଟ͕͘Googler or Xoogler •

    ಺෦Ͱ͸ Blaze ͱݺ͹ΕΔπʔϧ͕͋ΔΒ͍͠ (2006ʙ) • େن໛ͳίʔυɾϢʔβʔϕʔεͰ΋εέʔϧͰ͖Δ • ෳ਺ͷݴޠΛෳ਺ͷϓϥοτϑΥʔϜʹ޲͚ͯϏϧυ͢Δ͜ͱ͕Ͱ͖Δ • Java / C / C++ / Python / Go / Objective-C / JavaScript / Rust / protobuf 6
  3. Bazel • Bazel ຊମ͸ Java • αʔόʔʗΫϥΠΞϯτ Ϟσϧ • ඇৗʹ֦ுੑ͕ߴ͍

    • Starlark ͱ͍͏ಠࣗݴޠͰઃఆϑΝΠϧ͕هड़͞ΕΔ • ϏϧυϧʔϧʹΫΤϦΛ࣮ߦ͢Δ͜ͱ͕Ͱ͖Δ • Ϗϧυ͕ಠࣗͷαϯυϘοΫεͷதͰߦΘΕΔ • ڧྗͳΩϟογϡػߏΛ͍࣋ͬͯΔ 7
  4. ߴ͍֦ுੑ • Go ΛίϯύΠϧ͢Δ͜ͱ΋Ͱ͖Ε͹ Python ΋Ͱ͖Δ • ίϯςφ΍ rpm/deb ύοέʔδΛ࡞੒͢Δ͜ͱ΋Ͱ͖Δ

    • Java ΍ C++ ͸ຊମʹؚ·ΕΔ͕ͦΕҎ֎͸෼཭͞Ε͍ͯΔ • Starlark ʢޙड़ʣͰϏϧυͷखॱΛίʔυʹམͱ͠ࠐΉ • ϦϙδτϦ಺Ͱ֦ுΛॻ͘͜ͱ΋Ͱ͖Δ • ࠷ॳ͸֦ுΛॻ͘͜ͱ͸͓͢͢Ί͠·ͤΜ 8
  5. Starlark • Python ͷΑ͏ͳݴޠʢPythonͰ͸ͳ͍ʣ • Python ͷεΫϦϓτͰ Makefile Λੜ੒͍ͯͨ͠ྺ࢙తܦҢ •

    Ϗϧυϧʔϧ͸͢΂ͯ Starlark Ͱهड़͢Δ • ͨͩ͠ϏϧυϧʔϧͰ Starlark ͷจ๏͕͢΂ͯ࢖͑ΔΘ͚Ͱ͸ͳ͍ • ϏϧυΛهड़͢Δ͜ͱΛ໨తʹσβΠϯ͞Ε͍ͯΔ 9
  6. Starlark • จ๏͸ Python3 ʹΠϯεύΠΞ͞Ε͍ͯΔ • Ϗϧυͷ֦ு΋ Starlark Ͱهड़͢Δ 10

    def fizz_buzz(n): for i in range(1, n + 1): s = "" if i % 3 == 0: s += "Fizz" if i % 5 == 0: s += "Buzz" print(s if s else i) fizz_buzz(20) https://docs.bazel.build/versions/master/skylark/language.html
  7. ΫΤϦ • ఆٛͨ͠Ϗϧυϧʔϧશମʹରͯ͠ΫΤϦΛ࣮ߦͰ͖Δ • ࣮ߦՄೳͳόΠφϦΛఆ͍ٛͯ͠Δύοέʔδ • bazel query 'kind(go_binary, //...)'

    • ͋Δύοέʔδʹґଘ͍ͯ͠ΔύοέʔδͷϦετ • bazel query 'rdeps(...,)' • ϦϙδτϦ഑ԼͷίϯςφΠϝʔδΛ͢΂ͯpush͢ΔϫϯϥΠφʔ • bazel query 'kind(container_push, //...)' | xargs -n1 bazel run 11
  8. ґଘؔ܎ͷهड़ 15 gocon20 lib/logger pflag cmd/gocon20 lib/http lib/fluentd lib/file •

    ੒Ռ෺ΛಘΔͨΊʹඞཁͳೖྗΛ໌ه͢Δ • ೖྗ͸੒Ռ෺Ͱ͋ͬͯ΋Α͍ • ॥؀ࢀরΛ࡞Δ͜ͱ͸Ͱ͖ͳ͍
  9. ґଘؔ܎ͷهड़ 16 sh_binary( name = "sh", srcs = ["hello.sh"], deps

    = [":world"], ) sh_library( name = "world", srcs = ["world.sh"] ) Target ग़ྗͷ୯Ґ
  10. ґଘؔ܎ͷهड़ 17 sh_binary( name = "sh", srcs = ["hello.sh"], deps

    = [":world"], ) sh_library( name = "world", srcs = ["world.sh"] ) λʔήοτ໊
  11. ґଘؔ܎ͷهड़ 18 sh_binary( name = "sh", srcs = ["hello.sh"], deps

    = [":world"], ) sh_library( name = "world", srcs = ["world.sh"] ) ೖྗ
  12. ґଘؔ܎ͷهड़ 19 sh_binary( name = "sh", srcs = ["hello.sh"], deps

    = [":world"], ) sh_library( name = "world", srcs = ["world.sh"] ) ґଘ
  13. Visualize 20 //cmd/goimports:goimports //cmd/goimports:goimports_lib //cmd/goimports:goimports_gc.go //cmd/goimports:doc.go //cmd/goimports:goimports.go //internal/gocommand:gocommand //internal/imports:imports //vendor/golang.org/x/mod/semver:semver

    //internal/gocommand:vendor.go //internal/gocommand:invoke.go //internal/event:event //internal/imports:imports.go //internal/imports:zstdlib.go //internal/imports:mod.go //internal/imports:mod_cache.go //internal/imports:fix.go //internal/imports:sortimports.go //go/ast/astutil:astutil //internal/gopathwalk:gopathwalk //vendor/golang.org/x/mod/module:module //go/ast/astutil:rewrite.go //go/ast/astutil:util.go //go/ast/astutil:enclosing.go //go/ast/astutil:imports.go //internal/gopathwalk:walk.go //internal/fastwalk:fastwalk //vendor/golang.org/x/mod/module:module.go //vendor/golang.org/x/xerrors:xerrors //vendor/golang.org/x/mod/semver:semver.go //vendor/golang.org/x/xerrors:wrap.go //vendor/golang.org/x/xerrors:errors.go //vendor/golang.org/x/xerrors:doc.go //vendor/golang.org/x/xerrors:frame.go //vendor/golang.org/x/xerrors:format.go //vendor/golang.org/x/xerrors:adaptor.go //vendor/golang.org/x/xerrors:fmt.go //vendor/golang.org/x/xerrors/internal:internal //vendor/golang.org/x/xerrors/internal:internal.go //internal/fastwalk:fastwalk_dirent_namlen_bsd.go //internal/fastwalk:fastwalk_dirent_fileno.go //internal/fastwalk:fastwalk_dirent_ino.go //internal/fastwalk:fastwalk.go //internal/fastwalk:fastwalk_unix.go //internal/fastwalk:fastwalk_portable.go //internal/fastwalk:fastwalk_dirent_namlen_linux.go //internal/event:doc.go //internal/event:event.go //internal/event/core:core //internal/event/keys:keys //internal/event/label:label //internal/event/core:export.go //internal/event/core:event.go //internal/event/core:fast.go //internal/event/keys:keys.go //internal/event/keys:standard.go //internal/event/label:label.go
  14. rules_go • https://github.com/bazelbuild/rules_go • Go ΛϏϧυ͢ΔͨΊͷ֦ு • go_binary: ࣮ߦϑΝΠϧΛఆٛ͢Δ •

    go_library: ଞύοέʔδ͔ΒϦϯΫ͞ΕΔύοέʔδΛఆٛ͢Δ • go_test: ςετΛఆٛ͢Δ • ͜ΕΒͷϧʔϧ͸ιʔεϑΝΠϧ͔Βࣗಈੜ੒Մೳ • GoͷϥϯλΠϜ͸Ϗϧυϧʔϧͷ಺෦Ͱऔಘ͢ΔͷͰೖΕΔඞཁ͸ͳ͍ 24
  15. • ֎෦΁ͷґଘΛهड़͢ΔϑΝΠϧ Step 1: WORKSPACE 27 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name

    = "io_bazel_rules_go", sha256 = "b725e6497741d7fc2d55fcc29a276627d10e43fa5d0bb692692890ae30d98d00", urls = [ "https://github.com/bazelbuild/rules_go/releases/download/v0.24.3/rules_go-v0.24.3.tar.gz", ], ) http_archive( name = "bazel_gazelle", sha256 = "72d339ff874a382f819aaea80669be049069f502d6c726a07759fdca99653c48", urls = [ "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.22.1/bazel-gazelle-v0.22.1.tar.gz", ], ) load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies") go_rules_dependencies() go_register_toolchains() gazelle_dependencies()
  16. • Ϗϧυͷ੒Ռ෺ʹ͍ͭͯఆٛ͢Δ • ϦϙδτϦͷτοϓʹ࣍ͷϑΝΠϧΛ࡞Δ • ͜ͷϑΝΠϧ͚ͩ͸ࣗΒͷखͰ࡞Δඞཁ͕͋Δ Step 2: BUILD.bazel 28

    load("@bazel_gazelle//:def.bzl", "gazelle") # gazelle:prefix go.f110.dev/bazel-example # gazelle:proto disable_global gazelle(name = "gazelle")
  17. Step 3: gazelle update • BUILD.bazel Λࣗಈੜ੒ • bazel run

    //:gazelle -- update • ੜ੒͞Εͨ BUILD.bazel ͸͢΂ͯίϛοτ͢Δ 29
  18. ։ൃΛଓ͚Δ • Step 1: Write code • Step 2: BUILD.bazel

    Λߋ৽͢Δ • bazel run //:gazelle -- update • ґଘؔ܎ʹมߋ͕ͳ͚Ε͹ෆཁ • Step 3: ࣮ߦ͢Δ • bazel run //cmd/gocon20 30
  19. ґଘϥΠϒϥϦͷ؅ཧํ๏ • ؅ཧ͢Δํ๏͸2ͭ 1. WORKSPACE Ͱ؅ཧ͢Δํ๏ • gazelle update-repos -from_file=go.mod

    Ͱ WORKSPACE ϑΝΠϧΛߋ৽͢Δ 2. vendor σΟϨΫτϦΛϦϙδτϦʹؚΊΔ • vendor σΟϨΫτϦͰ؅ཧ͢Δํ๏͕͓͢͢Ί • bazel run @go_sdk//:bin/go -- mod vendor • WORKSPACE άϩʔόϧͳ֎෦ґଘ͕૿͑Δͱ؅ཧ͕େมʹͳΔ 31
  20. BazelͰϏϧυ͢Δͱ͖ͷϙΠϯτ (1/2) • ೖྗʹରͯ͠ग़ྗ͕มΘΒͳ͍Α͏ʹ͢Δ • Ϗϧυ࣌ؒΛຒΊࠐΉͱϏϧυຖʹग़ྗ͕ҟͳΔͷͰ΍Βͳ͍ • Gitͷίϛοτϋογϡ΋ຒΊࠐΉඞཁ͸ͳ͍ • ಉ͡ίϛοτ͔Βಉ͡όΠφϦϑΝΠϧ͕ग़ྗ͞ΕΔͷͰಛఆ͢Δ͜ͱ͸Ͱ͖Δ

    • ιʔείʔυͷࣗಈੜ੒Λ͍ͯ͠Δ৔߹͸݁ՌΛϦϙδτϦʹίϛοτ͢Δ • αϯυϘοΫε಺Ͱੜ੒ͨ͠ίʔυʹରͯ͠IDE͕ΠϯσοΫε͠ͳ͍ͷͰิ׬Ͱ͖ͳ͍ • ґଘϥΠϒϥϦ͸ vendor σΟϨΫτϦ؅ཧʹ͢Δ 32
  21. ίϯςφΛ࡞Δ • rules_docker Λ࢖͏ • ίϯςφΛ࡞Δࡍʹ docker ͕ඞཁͳ͍ • Docker

    Desktop on Mac ͔Βͷղ์ • ݡ͍Ωϟογϡ • rpm/deb ύοέʔδΛίϯςφʹೖΕΑ͏ͱࢥ͏ͱ͍͢͝େม 35
  22. όΠφϦ͕1͚ͭͩͷίϯςφ • go_binary ͷ୅ΘΓʹ go_image Λ࢖͏ 36 go_binary( name =

    "gocon20", embed = [":gocon20_lib"], visibility = ["//visibility:public"], ) load("@io_bazel_rules_docker//go:image.bzl", "go_image") go_image( name = "gocon20_image", embed = [":gocon20_lib"], goos = "linux", goarch = "amd64", visibility = ["//visibility:public"], )
  23. Ϗϧυ • bazel build //cmd/gocon20:gocon20_image.tar • dockerʹϩʔυ͢Δ • docker load

    -i bazel-bin/cmd/gocon20/gocon20_image.tar 37 $ docker images | grep gocon20 REPOSITORY TAG IMAGE ID CREATED SIZE bazel/cmd/gocon20 gocon20_image 6a76b205225f 50 years ago 23.3MB λΠϜελϯϓ͸ඞͣ0ʹͳΔ
  24. όΠφϦҎ֎ΛؚΊ͍ͨ৔߹ • όΠφϦҎ֎ΛؚΜͩ tar ͱόΠφϦΛؚΜͩ tar ΛϚʔδ͢ΔΠϝʔδ 38 load("@rules_pkg//:pkg.bzl", "pkg_tar")

    load("@io_bazel_rules_docker//container:container.bzl", "container_image") pkg_tar( name = "files", srcs = ["gocon.txt"], package_dir = "/", ) pkg_tar( name = "bin", srcs = [ "//cmd/gocon20:gocon20", ], mode = "0755", package_dir = "/usr/local/bin", ) container_image( name = "image", base = "@com_google_distroless_base//image", tars = [ ":files", ":bin", ], )
  25. push • ϓϥΠϕʔτϨδετϦʹରͯ͠ίϯςφΠϝʔδͷPush΋Ͱ͖Δ • ࣄલʹ docker login Λ͓ͯ͘͠ 39 load("@io_bazel_rules_docker//container:container.bzl",

    "container_push") container_push( name = "push", format = "Docker", image = ":image", registry = "quay.io", repository = "f110/bazel-example", tag = "latest", ) $ bazel run //:push $ cat bazel-bin/push.digest sha256:d6b76f53f79f030e117ac49323c294fff9dfebec5d036708e5f009189ed39104
  26. ·ͱΊ • Bazel ΛϏϧυπʔϧʹ͢Δͱґଘؔ܎͕ΫϦΞʹͳΔ • Bazel Ͱ Go ͷϓϩάϥϜ΋ίϯςφ΋Ϗϧυ͢Δ͜ͱ͕Ͱ͖Δ •

    ՝୊΍ߟ͑ํͷ੾Γସ͕͑ඞཁͳͱ͜Ζ΋͋Δ͕ͦΕΒΛղܾͰ͖Δͱշద • ֶशίετͷߴ͞ • ґଘϥΠϒϥϦͷ؅ཧ • Ϗϧυશମ΋ιϑτ΢ΣΞͱΈͳ͢ߟ͑ํ 41
  27. Reference • https://bazel.build/ • https://www.pantsbuild.org/ • https://buck.build/ • https://please.build/index.html •

    https://blog.bazel.build/2017/03/21/design-of-skylark.html • https://github.com/bazelbuild/starlark • https://blog.bazel.build/2015/09/11/sandboxing.html • https://github.com/bazelbuild/rules_go • https://github.com/bazelbuild/bazel-gazelle • https://github.com/bazelbuild/rules_docker 42