Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Terraform, GitHub Actions, Cloud Buildでデータ基盤をPr...
Search
Shinichi Nakagawa
PRO
March 15, 2024
Technology
2
3.4k
Terraform, GitHub Actions, Cloud Buildでデータ基盤をProvisioningする / Data Platform provisioning for Google Cloud and Terraform
会社名は入ってますが100%個人の開発と趣味の話です.
Shinichi Nakagawa
PRO
March 15, 2024
Tweet
Share
More Decks by Shinichi Nakagawa
See All by Shinichi Nakagawa
生成AI時代におけるSREの進化とキャリア戦略 / Building an Embedded SRE team and my career
shinyorke
PRO
0
100
生成AIを活用した野球データ分析 - メジャーリーグ編 / Baseball Analytics for Gen AI
shinyorke
PRO
1
4.9k
ゼロから始めるSREの事業貢献 - 生成AI時代のSRE成長戦略と実践 / Starting SRE from Day One
shinyorke
PRO
2
4.7k
AI・LLM事業部のSREとタスクの自動運転
shinyorke
PRO
0
470
実践Dash - 手を抜きながら本気で作るデータApplicationの基本と応用 / Dash for Python and Baseball
shinyorke
PRO
2
3.5k
Cloud RunとCloud PubSubでサーバレスなデータ基盤2024 with Terraform / Cloud Run and PubSub with Terraform
shinyorke
PRO
9
4.2k
自らを強いエンジニアにするための3つの習慣 / I need to be myself, I can't be no one else
shinyorke
PRO
85
88k
阪神タイガース優勝のひみつ - Pythonでシュッと調べた件 / SABRmetrics for Python
shinyorke
PRO
1
1.5k
Pythonとクラウドと野球の推し活. / Baseball Data Platform for Python and Google Cloud
shinyorke
PRO
2
3k
Other Decks in Technology
See All in Technology
AI ReadyなData PlatformとしてのAutonomous Databaseアップデート
oracle4engineer
PRO
0
170
自動テストのコストと向き合ってみた
qa
0
110
定期的な価値提供だけじゃない、スクラムが導くチームの共創化 / 20251004 Naoki Takahashi
shift_evolve
PRO
3
300
10年の共創が示す、これからの開発者と企業の関係 ~ Crossroad
soracom
PRO
1
170
"複雑なデータ処理 × 静的サイト" を両立させる、楽をするRails運用 / A low-effort Rails workflow that combines “Complex Data Processing × Static Sites”
hogelog
3
1.9k
Trust as Infrastructure
bcantrill
0
330
多野優介
tanoyusuke
1
420
Oracle Cloud Infrastructure:2025年9月度サービス・アップデート
oracle4engineer
PRO
0
390
生成AIを活用したZennの取り組み事例
ryosukeigarashi
0
200
後進育成のしくじり〜任せるスキルとリーダーシップの両立〜
matsu0228
6
2.3k
AI駆動開発を推進するためにサービス開発チームで 取り組んでいること
noayaoshiro
0
160
組織観点からIAM Identity CenterとIAMの設計を考える
nrinetcom
PRO
1
170
Featured
See All Featured
Art, The Web, and Tiny UX
lynnandtonic
303
21k
RailsConf 2023
tenderlove
30
1.2k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
9
580
Documentation Writing (for coders)
carmenintech
75
5k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
54
3k
Git: the NoSQL Database
bkeepers
PRO
431
66k
Agile that works and the tools we love
rasmusluckow
331
21k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
2.6k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Transcript
Terraform, GitHub Actions, Cloud BuildͰ σʔλج൫ͷProvisioningΛͬͯΈͨ. αʔόϨεΞʔΩςΫνϟͰMicro ServiceԽͨ͠ੈքઢͷDevOpsతͳߟ. Shinichi Nakagawa
2024/03/15 Jagu'e'r Cloud Native #13 ϋΠϒϦου Meetup
໔ࣄ߲ • ͜ͷࣄྫࢲʢத৳Ұʣݸਓͷझຯ։ൃ͔Βੜ·ΕͨߟͰ͢. • ॴଐ৫ɾاۀɾஂମΛද͢ΔൃݴɾҙݟͰ͍͟͝·ͤΜ. • ॴଐ৫ɾاۀ͓ΑͼͦͷεςʔΫϗϧμʔʹର͢Δ࣭ɾҙݟɾ ͍߹Θ͓ͤΑͼۀ༰ʹҰ͓͑͢Δ͜ͱͰ͖·ͤΜ. • εϥΠυͷը૾ݪଇʮ͍Β͢ͱʯΛར༻͍ͯ͠·͢.
※ʮ͍Β͢ͱʯҎ֎ͷը૾ग़యɾҾ༻ݩΛهࡌ͍ͯ͠·͢.
Who am I ? ʢ͓લ୭Α?ʣ • Shinichi Nakagawaʢத৳Ұʣ / @shinyorkeʢSNSʣ
• ΞΫηϯνϡΞגࣜձࣾϚωδϟʔ • ΫϥυΤϯδχΞϦϯά/SREʢຊ৬ʣ • ʮGoogle Cloud Partner Top Engineer 2024ʯड • AI, ML, Dataplatform, Baseballʢաڈͷࣄ+झຯಛٕʣ • ʮLean Baseballʯͱ͍͏ݸਓϒϩάͰ ʮٕज़ʯʮٿʯʮΩϟϦΞʯʹ͍ͭͯ৭ʑॻ͍ͯ·͢. https://shinyorke.hatenablog.com/
ຊͷ͓͠ͳ͕͖ʢ͢͜ͱʣ • σʔλج൫ΛTerraformͰؙͬͱIaCʢInfrastructure as Codeʣ. • GitHub ActionsͰGoogle CloudΛDevOpsͯ͠Έͨ. Cloud
RunͰ࡞ͬͨMicro ServiceΛͯ͠IaC&DevOpsͨ͠Ͱ͢. ※͜ͷࣄྫۀͰͳ͘झຯͰ͢ʢ͕࣮Ͱ͑Δͣʣ.
ٿσʔλੳج൫ ※ݸਓ։ൃͷͰ͢ • झຯͷٿΛຊ֨తʹσʔλͰݟΔͨΊͷج൫ΛຊؾͰ ࡞͍ͬͯ·͢ʢٕज़తͳݕূΛ݉Ͷͯʣ. • ಛʹϝδϟʔϦʔάʮBaseball Savantʯͱ͍͏ެࣜͷ σʔλαΠτ͕͋Γ, ϑΝϯͷզʑͰCSVܗࣜͰσʔλ
͕खʹೖΓ·͢&σʔλͷத͕݁ߏ͍͜͠Ͱ͢. • ͦͷʮ͍͜͠CSVσʔλʯΛऔΓࠐΈ, ੳɾղੳΛ ߦ͏ϝδϟʔϦʔάੳΞϓϦΛ2022͔Β։ൃɾӡ༻. ຖϦΞʔΩςΫνϟʢ࡞Γ͠ʣ͍ͯ͠·͢, ࠓ Cloud Runͷ৽ػೳʢCloud Run JobsʣͰ࡞Γ͠. ੳج൫ͷΞϓϦέʔγϣϯʢ2022൛ʣ ©Shinichi Nakagawa ©Shinichi Nakagawa
ٿσʔλੳج൫શମ૾ʢ2024൛ʣ ©Shinichi Nakagawa
σʔλج൫ͷ֤ػೳͰͬͯΔϞϊͱղઆ .JDSP4FSWJDFPSׂ (PPHMF$MPVE4FSWJDF ֓ཁ ࣮ݴޠɾ'8 5JNFS $MPVE4DIFEVMFS 5SJHHFSΛఆظ࣮ߦ ˞γʔζϯ։࢝ޙຖ DSPOࣜͰͷ5JNFS
5SJHHFS 8PSL fl PX $MPVE3VO+PCT &YQPSUFS*NQPSUFSΛ࣮ߦ ͢Δ5SJHHFSͱͳΔόον 1SFGFDU 1ZUIPO Ͱ࣮ -VJHJ ଞͷ࣮Ͱ0, &YQPSUFS $MPVE3VO #BTFCBMM4BWBOUΫΤϦΛ࣮ ߦ͠$47Λμϯϩʔυ (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %BUBMBLF $MPVE4UPSBHF &YQPSUFS͕ग़ྗͨ͠$47Λ ແՃͷੜσʔλͱͯ͠อ ͳ͠ *NQPSUFS $MPVE3VO $47σʔλΛνΣοΫ͠ #JH2VFSZʹ&YQPSU (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %8) #JH2VFSZ ੳऀPSΞϓϦ͔Β͏ %BUB8BSF)PVTF 42-Ͱ7JFXΛ࡞ͬͯӡ༻
.JDSP4FSWJDFPSׂ (PPHMF$MPVE4FSWJDF ֓ཁ ࣮ݴޠɾ'8 5JNFS $MPVE4DIFEVMFS 5SJHHFSΛఆظ࣮ߦ ˞γʔζϯ։࢝ޙຖ DSPOࣜͰͷ5JNFS 5SJHHFS
8PSL fl PX $MPVE3VO+PCT &YQPSUFS*NQPSUFSΛ࣮ߦ ͢Δ5SJHHFSͱͳΔόον 1SFGFDU 1ZUIPO Ͱ࣮ -VJHJ ଞͷ࣮Ͱ0, &YQPSUFS $MPVE3VO #BTFCBMM4BWBOUΫΤϦΛ࣮ ߦ͠$47Λμϯϩʔυ (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %BUBMBLF $MPVE4UPSBHF &YQPSUFS͕ग़ྗͨ͠$47Λ ແՃͷੜσʔλͱͯ͠อ ͳ͠ *NQPSUFS $MPVE3VO $47σʔλΛνΣοΫ͠ #JH2VFSZʹ&YQPSU (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %8) #JH2VFSZ ੳऀPSΞϓϦ͔Β͏ %BUB8BSF)PVTF 42-Ͱ7JFXΛ࡞ͬͯӡ༻ σʔλج൫ͷ֤ػೳͰͬͯΔϞϊͱղઆ શ෦खಈͰઃఆ͢Δͱഁ͢Δະདྷ͕͍ͬͯ·͢ʂ
Micro ServiceʹTerraformඞཁʢॾઆ༗Γʣ. • ෳͷΞϓϦέʔγϣϯΛचͭͳ͗ʢϐλΰϥεΠονʣ͢Δ߹, TerraformͰͷIaC͓ͬͯ͜͏, খ͍͞γεςϜͰ. • ࠓճͷߏͰ·͋·͋ͷͷGoogle CloudͷServiceΛར༻. •
Cloud RunΞϓϦ͕3ݸ + Volume, LoggingͳͲͷݻ༗ઃఆ • Pub/Sub TopicͱSchema͕ͦΕͧΕ2ݸ • Cloud Schedulerͷઃఆ • Cloud StorageͷBucketͱΞΫηε੍ޚ • ͜ΕΒͱ৭ʑඥͮ͘ઃఆ, ಛʹService Account͓ΑͼIAM←͜Ε͕࠷ۂऀʢηΩϡϦςΟతʹʣ • ҎલCloud Console or GUIͰ࡞͍͕ͬͯͨഁ͢Δʢྫ: ʹ͕֮͑ແ͍Service Account, InstanceͳͲʣͷͰࠓ͔ΒIaCԽ. ઃܭͱ࣮ͷॳظίετʢओʹ࣌ؒͱΩϟονΞοϓʣ͋Δ͕, ࢿͨ͠΄͏͕ઈରʹྑ͍ʢͱ࣮ײ͍ͯ͠·͢ʣ.
Google CloudͰTerraformΛ࢝ΊΔ࣌ • Quick StartͳͲ, ެࣜυΩϡϝϯτʹsnippet͕͋ΔͷͰ͓͏ʂ • gcloud commandGUIઃఆͱͷฒͼʹTerraformͷsnippet͕ଘࡏ. •
ࢲެࣜυΩϡϝϯτͷTerraformͷsnippetΛmain.tfʹషΓ͚, ಈ࡞ ֬ೝ͠ͳ͕Βຊ൪༻ͷ࣮Λߦͬͨʢ͘͢͝ḿͬͨʣ. શͯͷυΩϡϝϯτʹ͋Δ༁Ͱແͦ͞͏͕ͩੵۃతʹ͓͏.
ʁʁʁʮ͍͍ײ͡ʹࣗಈԽͱDevOps͍ͨ͠ΜͰ͕͢Ͳ͏͢Ε🤔ʯ Enterprise͚Ͱ࠷ۙ͋Δͱࢥ͏͜ͷཁʹͲ͏Ԡ͑Δ͔!?
ʲʳGitHub Actions͔ΒTerraformΛͬͨ • ʮ໎ͬͨΒGitHub ActionsʯͱࢥͬͯͬͨΒҙ֎ͱ্ख͍͖͘·ͨ͠. • ͬ͘͟Γॻ͘ͱҎԼͷखॱͰ࣮ݱ. 1. Workload Identityͷ४උ.
2. CDʢܧଓతσϓϩΠʣͱ࣮ͯ͠ߦ͢Δରͷܾఆ. 3. λεΫͷॱ൪ΛܾΊΔ. • Cloud RunΞϓϦέʔγϣϯͷ߹, ͜ͷ͚۠Ͱ͏·͘ߦͬͨ. • Workload Identity͓ΑͼΞϓϦͷService AccountʢSAʣൃߦGitHub ActionsͰͳ͘TerraformͰखಈ࣮ߦ. • SAͱ֤छϦιʔεͷBinding, ͦͷଞͷϦιʔεͷCDGitHub ActionsͰ࣮ߦ. • ArtifactʢDocker imageʣੜ -> DeployΛઈରతͳॱংͱͨ͠GitHub Actions Work fl owͷઃܭͱ࣮. ※͕͜͜ॏཁ
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
GitHub Actions + Terraform, Cloud Buildͷ • CI/CDεςʔδͷׂΛ໌֬Խ. • ʢͨΓલͰ͕͢ʣςετຖճࣗಈ࣮ߦ.
• Pull Request࣌ʹImage Buildͱterraform planͷ֬ೝ. • Deployʢterraform applyʣmainͷMergeͷΈ. • GitHub ActionsͰCI/CDͷϑϩʔΛ੍ޚ. • ͯ͢ͷΞϓϦέʔγϣϯͷArtifactΛCloud BuildͰ࡞ޙʹterraform planΛݩʹϨϏϡʔ. • GitHub ActionsͷneedsઃఆͱPull RequestͷonΠϕϯτͰ͍͍ײ͡ʹ੍ޚ. • ϨϏϡʔ༻ʹterraform planͷ݁ՌΛPull RequestͷίϝϯτʹͤΔ.
݁ͼ • ෳࡶʹͳΓ͕ͪͳMicro ServiceIaCԽ͠·͠ΐ͏, ΕΔͱେม. • GitHub ActionsͱCloud BuildͰTerraformͻͱ͕ඞཁ. ॳखͰIaCʢTerraformʣΛೖΕΔۤ࿑͋Γ·͕͢େมʹॏཁ.
͍͖ͳΓࣄͰΔͱେมͳͷͰԿ͔͠Β࿅शΛ͠·͠ΐ͏.
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠. Shinichi Nakagawa ©Shinichi Nakagawa