Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Terraform, GitHub Actions, Cloud Buildでデータ基盤をPr...
Search
Shinichi Nakagawa
PRO
March 15, 2024
Technology
2
3.2k
Terraform, GitHub Actions, Cloud Buildでデータ基盤をProvisioningする / Data Platform provisioning for Google Cloud and Terraform
会社名は入ってますが100%個人の開発と趣味の話です.
Shinichi Nakagawa
PRO
March 15, 2024
Tweet
Share
More Decks by Shinichi Nakagawa
See All by Shinichi Nakagawa
AI・LLM事業部のSREとタスクの自動運転
shinyorke
PRO
0
380
実践Dash - 手を抜きながら本気で作るデータApplicationの基本と応用 / Dash for Python and Baseball
shinyorke
PRO
2
2.8k
Cloud RunとCloud PubSubでサーバレスなデータ基盤2024 with Terraform / Cloud Run and PubSub with Terraform
shinyorke
PRO
9
3.9k
自らを強いエンジニアにするための3つの習慣 / I need to be myself, I can't be no one else
shinyorke
PRO
82
84k
阪神タイガース優勝のひみつ - Pythonでシュッと調べた件 / SABRmetrics for Python
shinyorke
PRO
1
1.4k
Pythonとクラウドと野球の推し活. / Baseball Data Platform for Python and Google Cloud
shinyorke
PRO
2
2.9k
月額コーヒー3.34杯分のコストでオオタニサンの活躍を見守るデータ基盤のはなし / Pyhack Con
shinyorke
PRO
2
510
俺のDXを実現するためのサーバレスなデータ基盤開発と運用 / Serverless Data Platform and Baseball
shinyorke
PRO
5
12k
機械学習エンジニアが目指すキャリアパスとその実話 / My Journey to Become a ML Engineer
shinyorke
PRO
10
18k
Other Decks in Technology
See All in Technology
ローカル環境でAIを動かそう!
falken
PRO
1
140
AIのための オンボーディングドキュメントを整備する - hirotea
hirotea
9
2.2k
CloudTrailも、GuardDutyも、VPC Flow logsも… ログ多すぎ問題の整理術
nikuyoshi
5
610
GigaViewerにおけるMackerel APM導入の裏側
7474
0
390
Postman AI エージェントビルダー最新情報
nagix
0
170
Things you never dared to ask about LLMs — v2
glaforge
1
460
CSS polyfill とその未来
ken7253
0
130
プロジェクトマネジメント実践論|現役エンジニアが語る!~チームでモノづくりをする時のコツとは?~
mixi_engineers
PRO
3
140
Streamline Cloud-Native App Development Using CDEs
saeedzf
0
640
SmartHRの複数のチームにおけるMCPサーバーの活用事例と課題
yukisnow1823
2
1k
Scale Security Programs with Scorecarding
ramimac
0
370
君だけのオリジナル async / await を作ろう / TSKaigi 2025
susisu
17
13k
Featured
See All Featured
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
180
53k
The Invisible Side of Design
smashingmag
299
50k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
5
610
Unsuck your backbone
ammeep
671
58k
How STYLIGHT went responsive
nonsquared
100
5.6k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
A designer walks into a library…
pauljervisheath
205
24k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
34
2.3k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Code Review Best Practice
trishagee
68
18k
Adopting Sorbet at Scale
ufuk
76
9.4k
Transcript
Terraform, GitHub Actions, Cloud BuildͰ σʔλج൫ͷProvisioningΛͬͯΈͨ. αʔόϨεΞʔΩςΫνϟͰMicro ServiceԽͨ͠ੈքઢͷDevOpsతͳߟ. Shinichi Nakagawa
2024/03/15 Jagu'e'r Cloud Native #13 ϋΠϒϦου Meetup
໔ࣄ߲ • ͜ͷࣄྫࢲʢத৳Ұʣݸਓͷझຯ։ൃ͔Βੜ·ΕͨߟͰ͢. • ॴଐ৫ɾاۀɾஂମΛද͢ΔൃݴɾҙݟͰ͍͟͝·ͤΜ. • ॴଐ৫ɾاۀ͓ΑͼͦͷεςʔΫϗϧμʔʹର͢Δ࣭ɾҙݟɾ ͍߹Θ͓ͤΑͼۀ༰ʹҰ͓͑͢Δ͜ͱͰ͖·ͤΜ. • εϥΠυͷը૾ݪଇʮ͍Β͢ͱʯΛར༻͍ͯ͠·͢.
※ʮ͍Β͢ͱʯҎ֎ͷը૾ग़యɾҾ༻ݩΛهࡌ͍ͯ͠·͢.
Who am I ? ʢ͓લ୭Α?ʣ • Shinichi Nakagawaʢத৳Ұʣ / @shinyorkeʢSNSʣ
• ΞΫηϯνϡΞגࣜձࣾϚωδϟʔ • ΫϥυΤϯδχΞϦϯά/SREʢຊ৬ʣ • ʮGoogle Cloud Partner Top Engineer 2024ʯड • AI, ML, Dataplatform, Baseballʢաڈͷࣄ+झຯಛٕʣ • ʮLean Baseballʯͱ͍͏ݸਓϒϩάͰ ʮٕज़ʯʮٿʯʮΩϟϦΞʯʹ͍ͭͯ৭ʑॻ͍ͯ·͢. https://shinyorke.hatenablog.com/
ຊͷ͓͠ͳ͕͖ʢ͢͜ͱʣ • σʔλج൫ΛTerraformͰؙͬͱIaCʢInfrastructure as Codeʣ. • GitHub ActionsͰGoogle CloudΛDevOpsͯ͠Έͨ. Cloud
RunͰ࡞ͬͨMicro ServiceΛͯ͠IaC&DevOpsͨ͠Ͱ͢. ※͜ͷࣄྫۀͰͳ͘झຯͰ͢ʢ͕࣮Ͱ͑Δͣʣ.
ٿσʔλੳج൫ ※ݸਓ։ൃͷͰ͢ • झຯͷٿΛຊ֨తʹσʔλͰݟΔͨΊͷج൫ΛຊؾͰ ࡞͍ͬͯ·͢ʢٕज़తͳݕূΛ݉Ͷͯʣ. • ಛʹϝδϟʔϦʔάʮBaseball Savantʯͱ͍͏ެࣜͷ σʔλαΠτ͕͋Γ, ϑΝϯͷզʑͰCSVܗࣜͰσʔλ
͕खʹೖΓ·͢&σʔλͷத͕݁ߏ͍͜͠Ͱ͢. • ͦͷʮ͍͜͠CSVσʔλʯΛऔΓࠐΈ, ੳɾղੳΛ ߦ͏ϝδϟʔϦʔάੳΞϓϦΛ2022͔Β։ൃɾӡ༻. ຖϦΞʔΩςΫνϟʢ࡞Γ͠ʣ͍ͯ͠·͢, ࠓ Cloud Runͷ৽ػೳʢCloud Run JobsʣͰ࡞Γ͠. ੳج൫ͷΞϓϦέʔγϣϯʢ2022൛ʣ ©Shinichi Nakagawa ©Shinichi Nakagawa
ٿσʔλੳج൫શମ૾ʢ2024൛ʣ ©Shinichi Nakagawa
σʔλج൫ͷ֤ػೳͰͬͯΔϞϊͱղઆ .JDSP4FSWJDFPSׂ (PPHMF$MPVE4FSWJDF ֓ཁ ࣮ݴޠɾ'8 5JNFS $MPVE4DIFEVMFS 5SJHHFSΛఆظ࣮ߦ ˞γʔζϯ։࢝ޙຖ DSPOࣜͰͷ5JNFS
5SJHHFS 8PSL fl PX $MPVE3VO+PCT &YQPSUFS*NQPSUFSΛ࣮ߦ ͢Δ5SJHHFSͱͳΔόον 1SFGFDU 1ZUIPO Ͱ࣮ -VJHJ ଞͷ࣮Ͱ0, &YQPSUFS $MPVE3VO #BTFCBMM4BWBOUΫΤϦΛ࣮ ߦ͠$47Λμϯϩʔυ (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %BUBMBLF $MPVE4UPSBHF &YQPSUFS͕ग़ྗͨ͠$47Λ ແՃͷੜσʔλͱͯ͠อ ͳ͠ *NQPSUFS $MPVE3VO $47σʔλΛνΣοΫ͠ #JH2VFSZʹ&YQPSU (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %8) #JH2VFSZ ੳऀPSΞϓϦ͔Β͏ %BUB8BSF)PVTF 42-Ͱ7JFXΛ࡞ͬͯӡ༻
.JDSP4FSWJDFPSׂ (PPHMF$MPVE4FSWJDF ֓ཁ ࣮ݴޠɾ'8 5JNFS $MPVE4DIFEVMFS 5SJHHFSΛఆظ࣮ߦ ˞γʔζϯ։࢝ޙຖ DSPOࣜͰͷ5JNFS 5SJHHFS
8PSL fl PX $MPVE3VO+PCT &YQPSUFS*NQPSUFSΛ࣮ߦ ͢Δ5SJHHFSͱͳΔόον 1SFGFDU 1ZUIPO Ͱ࣮ -VJHJ ଞͷ࣮Ͱ0, &YQPSUFS $MPVE3VO #BTFCBMM4BWBOUΫΤϦΛ࣮ ߦ͠$47Λμϯϩʔυ (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %BUBMBLF $MPVE4UPSBHF &YQPSUFS͕ग़ྗͨ͠$47Λ ແՃͷੜσʔλͱͯ͠อ ͳ͠ *NQPSUFS $MPVE3VO $47σʔλΛνΣοΫ͠ #JH2VFSZʹ&YQPSU (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %8) #JH2VFSZ ੳऀPSΞϓϦ͔Β͏ %BUB8BSF)PVTF 42-Ͱ7JFXΛ࡞ͬͯӡ༻ σʔλج൫ͷ֤ػೳͰͬͯΔϞϊͱղઆ શ෦खಈͰઃఆ͢Δͱഁ͢Δະདྷ͕͍ͬͯ·͢ʂ
Micro ServiceʹTerraformඞཁʢॾઆ༗Γʣ. • ෳͷΞϓϦέʔγϣϯΛचͭͳ͗ʢϐλΰϥεΠονʣ͢Δ߹, TerraformͰͷIaC͓ͬͯ͜͏, খ͍͞γεςϜͰ. • ࠓճͷߏͰ·͋·͋ͷͷGoogle CloudͷServiceΛར༻. •
Cloud RunΞϓϦ͕3ݸ + Volume, LoggingͳͲͷݻ༗ઃఆ • Pub/Sub TopicͱSchema͕ͦΕͧΕ2ݸ • Cloud Schedulerͷઃఆ • Cloud StorageͷBucketͱΞΫηε੍ޚ • ͜ΕΒͱ৭ʑඥͮ͘ઃఆ, ಛʹService Account͓ΑͼIAM←͜Ε͕࠷ۂऀʢηΩϡϦςΟతʹʣ • ҎલCloud Console or GUIͰ࡞͍͕ͬͯͨഁ͢Δʢྫ: ʹ͕֮͑ແ͍Service Account, InstanceͳͲʣͷͰࠓ͔ΒIaCԽ. ઃܭͱ࣮ͷॳظίετʢओʹ࣌ؒͱΩϟονΞοϓʣ͋Δ͕, ࢿͨ͠΄͏͕ઈରʹྑ͍ʢͱ࣮ײ͍ͯ͠·͢ʣ.
Google CloudͰTerraformΛ࢝ΊΔ࣌ • Quick StartͳͲ, ެࣜυΩϡϝϯτʹsnippet͕͋ΔͷͰ͓͏ʂ • gcloud commandGUIઃఆͱͷฒͼʹTerraformͷsnippet͕ଘࡏ. •
ࢲެࣜυΩϡϝϯτͷTerraformͷsnippetΛmain.tfʹషΓ͚, ಈ࡞ ֬ೝ͠ͳ͕Βຊ൪༻ͷ࣮Λߦͬͨʢ͘͢͝ḿͬͨʣ. શͯͷυΩϡϝϯτʹ͋Δ༁Ͱແͦ͞͏͕ͩੵۃతʹ͓͏.
ʁʁʁʮ͍͍ײ͡ʹࣗಈԽͱDevOps͍ͨ͠ΜͰ͕͢Ͳ͏͢Ε🤔ʯ Enterprise͚Ͱ࠷ۙ͋Δͱࢥ͏͜ͷཁʹͲ͏Ԡ͑Δ͔!?
ʲʳGitHub Actions͔ΒTerraformΛͬͨ • ʮ໎ͬͨΒGitHub ActionsʯͱࢥͬͯͬͨΒҙ֎ͱ্ख͍͖͘·ͨ͠. • ͬ͘͟Γॻ͘ͱҎԼͷखॱͰ࣮ݱ. 1. Workload Identityͷ४උ.
2. CDʢܧଓతσϓϩΠʣͱ࣮ͯ͠ߦ͢Δରͷܾఆ. 3. λεΫͷॱ൪ΛܾΊΔ. • Cloud RunΞϓϦέʔγϣϯͷ߹, ͜ͷ͚۠Ͱ͏·͘ߦͬͨ. • Workload Identity͓ΑͼΞϓϦͷService AccountʢSAʣൃߦGitHub ActionsͰͳ͘TerraformͰखಈ࣮ߦ. • SAͱ֤छϦιʔεͷBinding, ͦͷଞͷϦιʔεͷCDGitHub ActionsͰ࣮ߦ. • ArtifactʢDocker imageʣੜ -> DeployΛઈରతͳॱংͱͨ͠GitHub Actions Work fl owͷઃܭͱ࣮. ※͕͜͜ॏཁ
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
GitHub Actions + Terraform, Cloud Buildͷ • CI/CDεςʔδͷׂΛ໌֬Խ. • ʢͨΓલͰ͕͢ʣςετຖճࣗಈ࣮ߦ.
• Pull Request࣌ʹImage Buildͱterraform planͷ֬ೝ. • Deployʢterraform applyʣmainͷMergeͷΈ. • GitHub ActionsͰCI/CDͷϑϩʔΛ੍ޚ. • ͯ͢ͷΞϓϦέʔγϣϯͷArtifactΛCloud BuildͰ࡞ޙʹterraform planΛݩʹϨϏϡʔ. • GitHub ActionsͷneedsઃఆͱPull RequestͷonΠϕϯτͰ͍͍ײ͡ʹ੍ޚ. • ϨϏϡʔ༻ʹterraform planͷ݁ՌΛPull RequestͷίϝϯτʹͤΔ.
݁ͼ • ෳࡶʹͳΓ͕ͪͳMicro ServiceIaCԽ͠·͠ΐ͏, ΕΔͱେม. • GitHub ActionsͱCloud BuildͰTerraformͻͱ͕ඞཁ. ॳखͰIaCʢTerraformʣΛೖΕΔۤ࿑͋Γ·͕͢େมʹॏཁ.
͍͖ͳΓࣄͰΔͱେมͳͷͰԿ͔͠Β࿅शΛ͠·͠ΐ͏.
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠. Shinichi Nakagawa ©Shinichi Nakagawa