Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of aut...
Search
moznion
August 18, 2018
Technology
21
7.2k
無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of autonomous and infinite scalable Jenkins with AWS
Kyotoなんか #4の資料です
moznion
August 18, 2018
Tweet
Share
More Decks by moznion
See All by moznion
RubyKaigi Hack Space in Tokyo & 函館最速 "予習" 会 / RubyKaigi Hack Space in Tokyo & The Fastest Briefing of RubyKaigi 2026 in Hakodate
moznion
1
130
地に足の付いた現実的な技術選定から魔力のある体験を得る『AIレシート読み取り機能』のケーススタディ / From Grounded Tech Choices to Magical UX: A Case Study of AI Receipt Scanning
moznion
6
4k
Chrome Extension Techniques from Hell
moznion
1
200
Simple組み合わせ村から大都会Railsにやってきた俺は / Coming to Rails from the Simple
moznion
4
7.6k
AIレシート読み取り機能をRuby on Rails on AWSで実現するLLMにまつわるアレコレ / AI-based receipt reading function powered by LLM on Ruby on Rails on AWS
moznion
3
890
Develop to Survive - YAPC::Hakodate 2024 Keynote
moznion
11
18k
これはPerl? それともRuby? クイズ〜〜〜〜〜!!!- Perl or Ruby Quiz
moznion
3
2.8k
Perl 5 OOP機構30年史 - Perl 5's OOP Mechanism over the past 30 years
moznion
1
1.1k
RuboCopのカスタムCopを書いてContributionしてみる - Contributing a Custom Cop to RuboCop: A Hands-on Experience
moznion
0
90
Other Decks in Technology
See All in Technology
VCpp Link and Library - C++ breaktime 2025 Summer
harukasao
0
210
RubyOnRailsOnDevin+α / DevinMeetupJapan#2
ginkouno
0
810
TODAY 看世界(?) 是我們在看扣啦!
line_developers_tw
PRO
0
750
CI/CDとタスク共有で加速するVibe Coding
tnbe21
0
220
OCI Oracle Database Services新機能アップデート(2025/03-2025/05)
oracle4engineer
PRO
1
190
Rubyで作る論理回路シミュレータの設計の話 - Kashiwa.rb #12
kozy4324
1
320
CIでのgolangci-lintの実行を約90%削減した話
kazukihayase
0
320
2025/6/21 日本学術会議公開シンポジウム発表資料
keisuke198619
2
440
讓測試不再 BB! 從 BDD 到 CI/CD, 不靠人力也能 MVP
line_developers_tw
PRO
0
740
ハノーバーメッセ2025座談会.pdf
iotcomjpadmin
0
130
菸酒生在 LINE Taiwan 的後端雙刀流
line_developers_tw
PRO
0
710
米国国防総省のDevSecOpsライフサイクルをAWSのセキュリティサービスとOSSで実現
syoshie
0
300
Featured
See All Featured
Music & Morning Musume
bryan
46
6.6k
Being A Developer After 40
akosma
90
590k
Product Roadmaps are Hard
iamctodd
PRO
53
11k
4 Signs Your Business is Dying
shpigford
184
22k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Making Projects Easy
brettharned
116
6.2k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
46
9.6k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Rails Girls Zürich Keynote
gr2m
94
14k
Docker and Python
trallard
44
3.4k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
52
2.8k
Transcript
ແݶʹεέʔϧ͢Δ্ʹ ࣗతͳJenkinsʹݟΔເ ~AWSἫ~ KyotoͳΜ͔ #4 @moznion
@moznion Software engineer Hatena Intern 2013
Jenkins
Jenkins - CI (Continuous Integration) Platform - Workflow - ศརͳcron
Why Jenkins?
Why Jenkins? - ͱͱ (Ұ෦ࠓ) CircleCIΛ͍ͬͯͨ - Job͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……
- ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ͍͑ͨ -
Why Jenkins? - ͱͱ (Ұ෦ࠓ) CircleCIΛ͍ͬͯͨ - Job͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……
- ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ͍͑ͨ - ͱͱJenkinsΛεέʔϧͤ͞Δํ๏Λ͍ͬͯͨ
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
ฒྻjob্͛ΔͷϚδَʂ
ฒྻjob্͛ΔͷϚδَ SlaveΛ͢ͷ؆୯ʂʂʂʂ
ฒྻjob্͛ΔͷϚδَ - slave͢ͷ؆୯ͳΜ͚ͩͲ…… - ୯७ʹཧίετ͕૿͑Δͧʂ - ʮ͜ͷslaveͩͱͳ͔ͥςετ͕ίέ·͢ʯ
ฒྻjob্͛ΔͷϚδَ - slave͢ͷ؆୯ͳΜ͚ͩͲ…… - ΦʔτεέʔϦϯάΛͪ·ͪ·͍ͬͯ͘ඞཁ͕͋Γ…… - ΠϯελϯεΛ༡͓ͤͯ࣌ؒ͘ແ͍ - CIΑ΄ͲϓϩδΣΫτཱ͕ͯࠐΜͰͳ͍ݶΓ৸ͯΔ͕࣌ؒ ࢧత
- ׂͱઃఆ͕͍͠ (ίπ͕͍Δ)
͏͏……
զʑ΄ͬͱ͍ͯແݶʹϏϧυ͕εέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……
ͦΜͳͷ…… ͋Δʂʂʂ
ͦΜͳͷ…… ͋Δʂʂʂ
AWS CodeBuild
AWS CodeBuild - ୯Ґͷ࣌ؒି͠ - ҙͷίϯςφΛ࣮ߦͰ͖Δ - ͦͷίϯςφ্ͰϏϧυΒςετΒ͢Δ - (LambdaΑΓ͕͔͔࣌ؒΔλεΫͷ࣮ߦڥʹ……)
AWS CodeBuild - Jenkins Plugin͕͋Δ (AWSۘ) - https://github.com/awslabs/aws-codebuild-jenkins-plugin - JenkinsͰड͚ͨτϦΨͰCodeBuildΛൃՐͰ͖Δ
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/ ࣮࣭ແݶʂʂʂʂ
ͨΒ͞ΕͨԸܙ - slave nodeͷϝϯςΒͣ - ΨϯΨϯฒྻϏϧυ͢Δͧʂʂʂ - ࣌ؒି͠ (= ͚ͬͨͩ)
ͷྉۚઃఆͳͷͰ ࣗͰΦʔτεέʔϧॲཧΛ͠ͳͯ͘Α͍
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
Master Instance͕ਆ֨Խ͢Δ
Master Instanceਆ֨Խ - Jenkinsͷmaster datamasterϚγϯͷ*ϩʔΧϧετϨʔδ*ʹ อଘ͞ΕΔ - Πϯελϯε͕ࢮΜͩΒऴΘΓ - ͔͠͠ࢮආ͚ΒΕͳ͍
- ͔ͩΒόοΫΞοϓΛऔͬͨΓ͢Δ - e.g. https://papix.hatenablog.com/entry/2016/02/09/101918 - όοΫΞοϓ͔ΒͷϦετΞ࣮ࡍ໘ - ָ͍͠ࣄͰͳ͍…… - μϯλΠϜग़Δ͠……
Master Instanceਆ֨Խ - ݁Ռͱͯ͠ຖଉࡂΛفΔʑ…… - ӡѱ͘Πϯελϯε͕ࢮΜͩΒσΠϦʔόοΫΞοϓ ͔ΒϦετΞ - σΠϦʔόοΫΞοϓͱࢮ͵ؒࡍͱͷࠩʹ͍ͭͯ ఘΊΔ͔͠ແ͍……
͏͏……
զʑৎͰεέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……
ͦΜͳͷ…… ͋Δʂʂʂ
ͦΜͳͷ…… ͋Δʂʂʂ
Amazon Elastic File System
AWS Elastic File System (EFS) - AmazonͷnfsΈ͍ͨͳͭ - Instanceʹmountͯ͠͏ -
EBS + nfsతͳ…… - ϘϦϡʔϜαΠζ͕ࣗಈͰ֦ுɾॖখ͢Δ - Disk fullΒͣͷੜ׆
AWS Elastic File System (EFS) - $JENKINS_HOMEΛEFSϘϦϡʔϜ্ʹ͢Δ - Πϯελϯε͕ࢮΜͰ৽͍͠Πϯελϯεʹ ͦͷϘϦϡʔϜΛϚϯτ͢Δ
- => ΠϯελϯεલੈͷهԱΛҾ͖ܧ͗෮׆ʂʂʂ
mount Working...
❌mount Instance down...
❌ New instance coming up...
❌ mount Works fine!!
AWS Elastic File System (EFS) - EBSͰྑ͍ͷͰʁ - ͍ʂɹEBSͷ΄͏͕͍҆͠ʂʂ -
ͨͩEFSϘϦϡʔϜࣗಈ֦ு͕خ͍͠ - (͏͔ͬΓdisk fullʹͳΒͳ͍) - ͋ͱEFSͩͱmulti masterΈ͍ͨͳ͜ͱͰ͖Δ (ޙड़)
Jenkins Multi Master - ଟͰ͖Δ (࣌ؒӡ༻ͯ͠ͳ͍) - ͔͠͠CodeBuildͱڞଘ͍ͯ͠ΔڥͰMulti Masterʹ͢Δ ϝϦοτ΄΅ͳ͘ͳ͍Ͱ͔͢ʁ
- σϓϩΠ࣌ʹҰॠ2masterΛཱ͓͍ͯͯͯɼݹ͍ํΛ ࡴ͢Έ͍ͨͳμϯλΠϜσϓϩΠ͚ͷ͍ํՄೳ
ͨΒ͞ΕͨԸܙ - ΧδϡΞϧʹΠϯελϯε͕ࢮΜͰ҆৺ʂ - ͱ͍͑όοΫΞοϓऔͬͯΔ - ݁Ռͱͯ͠ΠϯελϯεͷೖΕସָ͕͑ʹͳͬͨ - ετϨʔδࣗಈ֦ு͢ΔͷͰDisk fullΒͣ
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
ࣾձ: Jenkins͓͡͞ΜͷͳΓखෆ
Jenkinsͷӡ༻Λ୭͔ʹԡ͚͠ΔͷΛΊΔ - ୭Ͱӡ༻Ͱ͖ΔΑ͏ʹ͢Δ - શਓຽJenkins͓͡͞Μ/͓͞ΜԽ - σϓϩΠϝϯςۃྗলྗԽ͢Δ
Jenkinsͷӡ༻Λ୭͔ʹԡ͚͠ΔͷΛΊΔ - Amazon Elastic Beanstalk + Docker - Dockerfile +
Dockerrun.aws.jsonʹΑΔߏͷίʔυԽ - .ebextensionsʹΑΔࣗಈԽ - EFS mountͳͲ - https://github.com/moznion/jenkins_eb_efs_codebuild_example
- Elastic Beanstalkͷmanaged updateͰࣗಈతʹύον͕ͨΔ - ͍ͭͰʹΠϯελϯεͷೖΕସ͑ߦΘΕΔ - EFSڥԼͳͷͰΠϯελϯεೖΕସָ͑ʑࣗಈԽ - جຊతʹӡ༻์ஔͯ͠·͢
- EFSͷϘϦϡʔϜࣗಈ֦ுͷύϫʔͰ͋Δ ͨΒ͞ΕͨԸܙ
·ͱΊ
·ͱΊ - JenkinsͷslaveϊʔυΛAWS CodeBuildʹҕৡ͢Δ͜ͱͰ࣮࣭ ແݶʹฒྻϏϧυ͕εέʔϧ͢ΔΑ͏ʹͳΓ·ͨ͠ - JenkinsͷετϨʔδΛEFSʹ͢Δ͜ͱͰϚελʔσʔλͷ ϩόετԽΛߦ͍·ͨ͠ - Elastic
BeanstalkΛ༻͍ͯJenkinsΛӡ༻͢Δ͜ͱͰӡ༻ͷ ίετԽΛ͠·ͨ͠ - Α͔ͬͨͰ͢Ͷ
Q?
Wait...
͜ͷJenkinsʹ͕͋Δ - EFS͍ʂʂʂʂʂ - Jenkins͕git clone͢ΔͱϚδ͔͔࣌ؒΓ·͢ - git cloneΛճආ͢Δํ๏͍·ͷͱ͜Ζແ͍…… -
shallow cloneΛ༗ޮʹ͏ - sparse checkoutΛ༗ޮʹ͏ - ͳͲͷ͕ඞཁ - ͦͦCodeBuild͕git clone͢ΔΜͩ͠jenkins͠ͳͯ͘ ྑ͍Μ͚ͩͲ…… - ͋Δ͍EBSΛ͏……
ຊʹJenkinsඞཁʁ - ͭ͡AWS CodeBuild࠷ۙΊͬͪΌͰ͖͕Αͯ͘ɼ ୯ମͰे͑Δ - GitHubͷhookड͚ΕΔ͠ - Commit statusΛม͑Δػೳ·Ͱ͍͍ͭͯΔ
- ϏϧυཤྺҰཡͰ͖Δ͠ - CloudWatch LogsʹϏϧυͷϩάग़Δ - JenkinsෆཁͰʁʁʁʁɹ͜ΕͰेͰʁʁʁʁ - ͍
ຊʹJenkinsඞཁʁ - ࣗ༝ͳWebhookΛड͚Δඞཁ͕͋Δͱ͔…… - ͋ͱ௨ͱ͔Ͷʂ - ͋Δ͍ͬͱଞʹϏϧυύΠϓϥΠϯΛܨ ͛Δඞཁ͕͋Δͱ͔…… - ͦ͏͍͏ͱ͖ΜΓͰ͢
- (͋ͱ·͋Ұཡੑ͕Α͍) (ॾઆ͋Γ·͢)
Q?