Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Terraform, GitHub Actions, Cloud Buildでデータ基盤をPr...
Search
Shinichi Nakagawa
PRO
March 15, 2024
Technology
2
3.1k
Terraform, GitHub Actions, Cloud Buildでデータ基盤をProvisioningする / Data Platform provisioning for Google Cloud and Terraform
会社名は入ってますが100%個人の開発と趣味の話です.
Shinichi Nakagawa
PRO
March 15, 2024
Tweet
Share
More Decks by Shinichi Nakagawa
See All by Shinichi Nakagawa
AI・LLM事業部のSREとタスクの自動運転
shinyorke
PRO
0
290
実践Dash - 手を抜きながら本気で作るデータApplicationの基本と応用 / Dash for Python and Baseball
shinyorke
PRO
2
2.4k
Cloud RunとCloud PubSubでサーバレスなデータ基盤2024 with Terraform / Cloud Run and PubSub with Terraform
shinyorke
PRO
9
3.7k
自らを強いエンジニアにするための3つの習慣 / I need to be myself, I can't be no one else
shinyorke
PRO
82
83k
阪神タイガース優勝のひみつ - Pythonでシュッと調べた件 / SABRmetrics for Python
shinyorke
PRO
1
1.4k
Pythonとクラウドと野球の推し活. / Baseball Data Platform for Python and Google Cloud
shinyorke
PRO
2
2.8k
月額コーヒー3.34杯分のコストでオオタニサンの活躍を見守るデータ基盤のはなし / Pyhack Con
shinyorke
PRO
2
500
俺のDXを実現するためのサーバレスなデータ基盤開発と運用 / Serverless Data Platform and Baseball
shinyorke
PRO
5
12k
機械学習エンジニアが目指すキャリアパスとその実話 / My Journey to Become a ML Engineer
shinyorke
PRO
10
18k
Other Decks in Technology
See All in Technology
ドメインイベントを活用したPHPコードのリファクタリング
kajitack
2
1.1k
パスキーでのログインを 実装してみよう!
hibiki_cube
0
580
ペアプログラミングにQAが加わった!職能を超えたモブプログラミングの事例と学び
tonionagauzzi
1
130
caching_sha2_passwordのはなし
boro1234
0
200
17年のQA経験が導いたスクラムマスターへの道 / 17 Years in QA to Scrum Master
toma_sm
0
350
製造業の会計システムをDDDで開発した話
caddi_eng
3
900
React Server Componentは 何を解決し何を解決しないのか / What do React Server Components solve, and what do they not solve?
kaminashi
6
1.1k
Go の analysis パッケージで自作するリファクタリングツール
kworkdev
PRO
1
360
スケールアップ企業のQA組織のバリューを最大限に引き出すための取り組み
tarappo
4
860
SpannerとAurora DSQLの同時実行制御の違いに想いを馳せる
masakikato5
0
550
LINEギフトのLINEミニアプリアクセシビリティ改善事例
lycorptech_jp
PRO
0
210
Why Go?
xpmatteo
0
130
Featured
See All Featured
Build your cross-platform service in a week with App Engine
jlugia
229
18k
We Have a Design System, Now What?
morganepeng
51
7.5k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.8k
RailsConf 2023
tenderlove
29
1k
Embracing the Ebb and Flow
colly
84
4.6k
Visualization
eitanlees
146
15k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
7.1k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Producing Creativity
orderedlist
PRO
344
40k
Raft: Consensus for Rubyists
vanstee
137
6.8k
Measuring & Analyzing Core Web Vitals
bluesmoon
6
320
Art, The Web, and Tiny UX
lynnandtonic
298
20k
Transcript
Terraform, GitHub Actions, Cloud BuildͰ σʔλج൫ͷProvisioningΛͬͯΈͨ. αʔόϨεΞʔΩςΫνϟͰMicro ServiceԽͨ͠ੈքઢͷDevOpsతͳߟ. Shinichi Nakagawa
2024/03/15 Jagu'e'r Cloud Native #13 ϋΠϒϦου Meetup
໔ࣄ߲ • ͜ͷࣄྫࢲʢத৳Ұʣݸਓͷझຯ։ൃ͔Βੜ·ΕͨߟͰ͢. • ॴଐ৫ɾاۀɾஂମΛද͢ΔൃݴɾҙݟͰ͍͟͝·ͤΜ. • ॴଐ৫ɾاۀ͓ΑͼͦͷεςʔΫϗϧμʔʹର͢Δ࣭ɾҙݟɾ ͍߹Θ͓ͤΑͼۀ༰ʹҰ͓͑͢Δ͜ͱͰ͖·ͤΜ. • εϥΠυͷը૾ݪଇʮ͍Β͢ͱʯΛར༻͍ͯ͠·͢.
※ʮ͍Β͢ͱʯҎ֎ͷը૾ग़యɾҾ༻ݩΛهࡌ͍ͯ͠·͢.
Who am I ? ʢ͓લ୭Α?ʣ • Shinichi Nakagawaʢத৳Ұʣ / @shinyorkeʢSNSʣ
• ΞΫηϯνϡΞגࣜձࣾϚωδϟʔ • ΫϥυΤϯδχΞϦϯά/SREʢຊ৬ʣ • ʮGoogle Cloud Partner Top Engineer 2024ʯड • AI, ML, Dataplatform, Baseballʢաڈͷࣄ+झຯಛٕʣ • ʮLean Baseballʯͱ͍͏ݸਓϒϩάͰ ʮٕज़ʯʮٿʯʮΩϟϦΞʯʹ͍ͭͯ৭ʑॻ͍ͯ·͢. https://shinyorke.hatenablog.com/
ຊͷ͓͠ͳ͕͖ʢ͢͜ͱʣ • σʔλج൫ΛTerraformͰؙͬͱIaCʢInfrastructure as Codeʣ. • GitHub ActionsͰGoogle CloudΛDevOpsͯ͠Έͨ. Cloud
RunͰ࡞ͬͨMicro ServiceΛͯ͠IaC&DevOpsͨ͠Ͱ͢. ※͜ͷࣄྫۀͰͳ͘झຯͰ͢ʢ͕࣮Ͱ͑Δͣʣ.
ٿσʔλੳج൫ ※ݸਓ։ൃͷͰ͢ • झຯͷٿΛຊ֨తʹσʔλͰݟΔͨΊͷج൫ΛຊؾͰ ࡞͍ͬͯ·͢ʢٕज़తͳݕূΛ݉Ͷͯʣ. • ಛʹϝδϟʔϦʔάʮBaseball Savantʯͱ͍͏ެࣜͷ σʔλαΠτ͕͋Γ, ϑΝϯͷզʑͰCSVܗࣜͰσʔλ
͕खʹೖΓ·͢&σʔλͷத͕݁ߏ͍͜͠Ͱ͢. • ͦͷʮ͍͜͠CSVσʔλʯΛऔΓࠐΈ, ੳɾղੳΛ ߦ͏ϝδϟʔϦʔάੳΞϓϦΛ2022͔Β։ൃɾӡ༻. ຖϦΞʔΩςΫνϟʢ࡞Γ͠ʣ͍ͯ͠·͢, ࠓ Cloud Runͷ৽ػೳʢCloud Run JobsʣͰ࡞Γ͠. ੳج൫ͷΞϓϦέʔγϣϯʢ2022൛ʣ ©Shinichi Nakagawa ©Shinichi Nakagawa
ٿσʔλੳج൫શମ૾ʢ2024൛ʣ ©Shinichi Nakagawa
σʔλج൫ͷ֤ػೳͰͬͯΔϞϊͱղઆ .JDSP4FSWJDFPSׂ (PPHMF$MPVE4FSWJDF ֓ཁ ࣮ݴޠɾ'8 5JNFS $MPVE4DIFEVMFS 5SJHHFSΛఆظ࣮ߦ ˞γʔζϯ։࢝ޙຖ DSPOࣜͰͷ5JNFS
5SJHHFS 8PSL fl PX $MPVE3VO+PCT &YQPSUFS*NQPSUFSΛ࣮ߦ ͢Δ5SJHHFSͱͳΔόον 1SFGFDU 1ZUIPO Ͱ࣮ -VJHJ ଞͷ࣮Ͱ0, &YQPSUFS $MPVE3VO #BTFCBMM4BWBOUΫΤϦΛ࣮ ߦ͠$47Λμϯϩʔυ (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %BUBMBLF $MPVE4UPSBHF &YQPSUFS͕ग़ྗͨ͠$47Λ ແՃͷੜσʔλͱͯ͠อ ͳ͠ *NQPSUFS $MPVE3VO $47σʔλΛνΣοΫ͠ #JH2VFSZʹ&YQPSU (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %8) #JH2VFSZ ੳऀPSΞϓϦ͔Β͏ %BUB8BSF)PVTF 42-Ͱ7JFXΛ࡞ͬͯӡ༻
.JDSP4FSWJDFPSׂ (PPHMF$MPVE4FSWJDF ֓ཁ ࣮ݴޠɾ'8 5JNFS $MPVE4DIFEVMFS 5SJHHFSΛఆظ࣮ߦ ˞γʔζϯ։࢝ޙຖ DSPOࣜͰͷ5JNFS 5SJHHFS
8PSL fl PX $MPVE3VO+PCT &YQPSUFS*NQPSUFSΛ࣮ߦ ͢Δ5SJHHFSͱͳΔόον 1SFGFDU 1ZUIPO Ͱ࣮ -VJHJ ଞͷ࣮Ͱ0, &YQPSUFS $MPVE3VO #BTFCBMM4BWBOUΫΤϦΛ࣮ ߦ͠$47Λμϯϩʔυ (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %BUBMBLF $MPVE4UPSBHF &YQPSUFS͕ग़ྗͨ͠$47Λ ແՃͷੜσʔλͱͯ͠อ ͳ͠ *NQPSUFS $MPVE3VO $47σʔλΛνΣοΫ͠ #JH2VFSZʹ&YQPSU (Pͷ8FC"1* 1VC4VC5PQJD͔Βىಈ %8) #JH2VFSZ ੳऀPSΞϓϦ͔Β͏ %BUB8BSF)PVTF 42-Ͱ7JFXΛ࡞ͬͯӡ༻ σʔλج൫ͷ֤ػೳͰͬͯΔϞϊͱղઆ શ෦खಈͰઃఆ͢Δͱഁ͢Δະདྷ͕͍ͬͯ·͢ʂ
Micro ServiceʹTerraformඞཁʢॾઆ༗Γʣ. • ෳͷΞϓϦέʔγϣϯΛचͭͳ͗ʢϐλΰϥεΠονʣ͢Δ߹, TerraformͰͷIaC͓ͬͯ͜͏, খ͍͞γεςϜͰ. • ࠓճͷߏͰ·͋·͋ͷͷGoogle CloudͷServiceΛར༻. •
Cloud RunΞϓϦ͕3ݸ + Volume, LoggingͳͲͷݻ༗ઃఆ • Pub/Sub TopicͱSchema͕ͦΕͧΕ2ݸ • Cloud Schedulerͷઃఆ • Cloud StorageͷBucketͱΞΫηε੍ޚ • ͜ΕΒͱ৭ʑඥͮ͘ઃఆ, ಛʹService Account͓ΑͼIAM←͜Ε͕࠷ۂऀʢηΩϡϦςΟతʹʣ • ҎલCloud Console or GUIͰ࡞͍͕ͬͯͨഁ͢Δʢྫ: ʹ͕֮͑ແ͍Service Account, InstanceͳͲʣͷͰࠓ͔ΒIaCԽ. ઃܭͱ࣮ͷॳظίετʢओʹ࣌ؒͱΩϟονΞοϓʣ͋Δ͕, ࢿͨ͠΄͏͕ઈରʹྑ͍ʢͱ࣮ײ͍ͯ͠·͢ʣ.
Google CloudͰTerraformΛ࢝ΊΔ࣌ • Quick StartͳͲ, ެࣜυΩϡϝϯτʹsnippet͕͋ΔͷͰ͓͏ʂ • gcloud commandGUIઃఆͱͷฒͼʹTerraformͷsnippet͕ଘࡏ. •
ࢲެࣜυΩϡϝϯτͷTerraformͷsnippetΛmain.tfʹషΓ͚, ಈ࡞ ֬ೝ͠ͳ͕Βຊ൪༻ͷ࣮Λߦͬͨʢ͘͢͝ḿͬͨʣ. શͯͷυΩϡϝϯτʹ͋Δ༁Ͱແͦ͞͏͕ͩੵۃతʹ͓͏.
ʁʁʁʮ͍͍ײ͡ʹࣗಈԽͱDevOps͍ͨ͠ΜͰ͕͢Ͳ͏͢Ε🤔ʯ Enterprise͚Ͱ࠷ۙ͋Δͱࢥ͏͜ͷཁʹͲ͏Ԡ͑Δ͔!?
ʲʳGitHub Actions͔ΒTerraformΛͬͨ • ʮ໎ͬͨΒGitHub ActionsʯͱࢥͬͯͬͨΒҙ֎ͱ্ख͍͖͘·ͨ͠. • ͬ͘͟Γॻ͘ͱҎԼͷखॱͰ࣮ݱ. 1. Workload Identityͷ४උ.
2. CDʢܧଓతσϓϩΠʣͱ࣮ͯ͠ߦ͢Δରͷܾఆ. 3. λεΫͷॱ൪ΛܾΊΔ. • Cloud RunΞϓϦέʔγϣϯͷ߹, ͜ͷ͚۠Ͱ͏·͘ߦͬͨ. • Workload Identity͓ΑͼΞϓϦͷService AccountʢSAʣൃߦGitHub ActionsͰͳ͘TerraformͰखಈ࣮ߦ. • SAͱ֤छϦιʔεͷBinding, ͦͷଞͷϦιʔεͷCDGitHub ActionsͰ࣮ߦ. • ArtifactʢDocker imageʣੜ -> DeployΛઈରతͳॱংͱͨ͠GitHub Actions Work fl owͷઃܭͱ࣮. ※͕͜͜ॏཁ
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
ʲਤʳࠓճ࠾༻ͨ͠GitHub ActionsͷCI/CD Flow. Cloud BuildʹΑΔImage BuildͱPull Requestཱ͕͔ͯ͠ΒTerraform ApplyͷྲྀΕʹ. ©Shinichi Nakagawa
GitHub Actions + Terraform, Cloud Buildͷ • CI/CDεςʔδͷׂΛ໌֬Խ. • ʢͨΓલͰ͕͢ʣςετຖճࣗಈ࣮ߦ.
• Pull Request࣌ʹImage Buildͱterraform planͷ֬ೝ. • Deployʢterraform applyʣmainͷMergeͷΈ. • GitHub ActionsͰCI/CDͷϑϩʔΛ੍ޚ. • ͯ͢ͷΞϓϦέʔγϣϯͷArtifactΛCloud BuildͰ࡞ޙʹterraform planΛݩʹϨϏϡʔ. • GitHub ActionsͷneedsઃఆͱPull RequestͷonΠϕϯτͰ͍͍ײ͡ʹ੍ޚ. • ϨϏϡʔ༻ʹterraform planͷ݁ՌΛPull RequestͷίϝϯτʹͤΔ.
݁ͼ • ෳࡶʹͳΓ͕ͪͳMicro ServiceIaCԽ͠·͠ΐ͏, ΕΔͱେม. • GitHub ActionsͱCloud BuildͰTerraformͻͱ͕ඞཁ. ॳखͰIaCʢTerraformʣΛೖΕΔۤ࿑͋Γ·͕͢େมʹॏཁ.
͍͖ͳΓࣄͰΔͱେมͳͷͰԿ͔͠Β࿅शΛ͠·͠ΐ͏.
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠. Shinichi Nakagawa ©Shinichi Nakagawa