Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Next FRESH! Applications with Amazon ECS
Search
stormcat24
November 20, 2015
Programming
13
8.4k
Next FRESH! Applications with Amazon ECS
2015/11/20 JAWS-UGコンテナ支部 #3 #jawsug_ct
stormcat24
November 20, 2015
Tweet
Share
More Decks by stormcat24
See All by stormcat24
素早く賢く失敗するDeveloper Productivityの実現を目指して
stormcat24
4
4.3k
KubernetesのマニフェストをそれなりにCIしたい
stormcat24
4
1.2k
令和時代のSaaS開発
stormcat24
1
230
History in 5 years of CircleCI and CyberAgent
stormcat24
3
800
Kubernetes Handson Osaka
stormcat24
5
530
Kubernetes Handson
stormcat24
5
4.2k
DockerとKubernetesでアプリケーション開発にコンテナをフル活用!
stormcat24
0
290
Base Image Journey 2018
stormcat24
29
130k
kotlin-fest
stormcat24
13
17k
Other Decks in Programming
See All in Programming
事業フェーズの変化に対応する 開発生産性向上のゼロイチ
masaygggg
0
200
いまから追い上げる、Jetpack Compose トレーニング
nyafunta9858
0
590
今インフラ技術をイチから学び直すなら
yuhta28
1
140
『ドメイン駆動設計をはじめよう』中核の業務領域
masuda220
PRO
5
1k
GoのIteratorに詳しくなってしまう
inatonix
1
200
状態管理ライブラリZustandの導入から運用まで
k1tikurisu
3
470
実践!難読化ガイド
mitchan
0
200
ECMAScript、Web標準の型はどう管理されているか / How ECMAScript and Web standards types are maintained
petamoriken
3
390
LangGraphでのHuman-in-the-Loopの実装
os1ma
3
1.1k
connect-go で面倒くささと戦う / 2024-08-27 #newmo_layerx_go
izumin5210
2
650
Patched fetch did not work
quramy
4
390
What you can do with Ruby on WebAssembly
kateinoigakukun
0
170
Featured
See All Featured
Understanding Cognitive Biases in Performance Measurement
bluesmoon
26
1.3k
Web development in the modern age
philhawksworth
205
10k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
248
20k
Code Reviewing Like a Champion
maltzj
517
39k
Raft: Consensus for Rubyists
vanstee
135
6.5k
jQuery: Nuts, Bolts and Bling
dougneiner
61
7.4k
The Pragmatic Product Professional
lauravandoore
31
6.2k
How to Think Like a Performance Engineer
csswizardry
16
960
Designing with Data
zakiwarfel
98
5k
Navigating Team Friction
lara
183
13k
From Idea to $5000 a Month in 5 Months
shpigford
379
46k
Learning to Love Humans: Emotional Interface Design
aarron
270
40k
Transcript
Next FRESH! Applications with Amazon ECS 2015/11/20 JAWS-UGίϯςφࢧ෦ #3 #jawsug_ct
@stormcat24
Who? ‣ Akinori Yamada ( @stormcat24 ) ‣ http://stormcat.hatenablog.com/ ‣
CyberAgent, Inc ‣ Ameba౷ׅຊ෦ FRESHάϧʔϓ ‣ ServerSide / DevOps ‣ ͖ͳݴޠScala
Agenda ‣ AmebaFRESH! ‣ Architecture & ECS ‣ Blue Green
Deployment ‣ Diet Docker Image ‣ Local Development ‣ ecs-formation ‣ Others
AmebaFRESH!
‣ ੜ์ૹಈը৴ϓϥοτϑΥʔϜ ‣ جຊແྉ ‣ PCϒϥβ / iOS / Android
‣ εϚϗφΠζυ͞ΕͨUI ‣ ߴ࣭ͳಈը৴ ‣ 201512݄Ұൠެ։༧ఆ ‣ ݱࡏΫϩʔζυެ։த ‣ ※AbemaTVͱผͷαʔϏεͰ͢YO
None
None
None
None
‣ 2015/04͔ΒϓϩδΣΫτ։࢝ ‣ ϓϩδΣΫτϝϯόʔ30໊ ‣ Serverside ☓ 6 ‣ Frontend
☓ 6 ‣ iOS ☓ 4 ‣ Android ☓ 4 ‣ Designer ☓ 3
‣ Movie & Broadcasting ‣ RTMP Publishing ‣ HTTP Live
Streaming (HLS)
‣ Frontend ‣ Node.js v4 ‣ TypeScript1.6 ‣ React /
Flux ‣ SPA + SSR ‣ iOS ‣ Swift2.0 ‣ Android ‣ Kotlin
‣ Serverside & Infrastructure ‣ Amazon Web Services (AWS) ‣
Go1.5 ‣ Docker1.9.0 ‣ Microservices ‣ RESTful API (goji)
‣ Tools ‣ Slack + Hubot (ChatOps) ‣ Github Enterprise
‣ CircleCI (Enterprise) ‣ Crashlytics ‣ JIRA Agile ‣ Mackerel ‣ ࡳଋ
Architecture & ECS
Architecture ‣ ࢥ ‣ جຊํ ‣ Task Definitionͷߏ ‣ ClusterͱServiceͷѻ͍
ࢦ͍ͯ͠Δ͜ͱ ‣ ۃྗϝϯςΛೖΕͳ͍ ‣ θϩμϯλΠϜϦϦʔε → Blue Green Deployment ‣
Πϯϑϥ͍ࣺͯͯೖΕସ͑Δ → Immutable Infrastructure ‣ ૉૣ͘සൟͳϦϦʔε ‣ γεςϜݶΓͳ͘ૄ݁߹ͷํ͕ྑ͍ → Microservices ‣ ίϯςφͩ → Docker
Microservices ‣ ಛఆͷ։ൃݴޠʹґଘ͠ͳ͍ʢ͖Δ͠ʣ ‣ ௨৴ϓϩτίϧ ‣ RESTFul API (HTTP) ‣
ͦͷ͏ͪgRPC(HTTP2)͍ͨ͠ ‣ αʔϏεͷཻΛఆΊΔͷΓ͍͠ ‣ γεςϜతͳυϝΠϯྖҬͰ͚ΔͷҰͭͷࢦඪ ‣ ޙ͔ΒผαʔϏεͱͯ͠Γམͱ͢OK
Infrastructure ‣ AWS + EC2 Container Service(ECS)Λ࠾༻ ‣ ͪΐ͏Ͳٕज़ݕূதʹECSͷ౦ژϦʔδϣϯ͕དྷͨ ‣
࠷ݶͷίϯςφߏཧͱεέδϡʔϦϯά͕Ͱ͖Εे ‣ ECSҎ֎ʹ͍͍ͨͷ͕͋ͬͨ͠ ‣ Lambda ‣ RDS Aurora
جຊํ ‣ MicroservicesຖͰίϯςφΛߏ͢ΔTask DefinitionΛͭ͘Δ ‣ 1ECS Clusterʹ͖ͭ1Service ‣ 1Clusterʹ1ͭͷAutoScaling Group
‣ ಈը৴αʔόಛघͳͷͰྫ֎ ‣ Clusterؒ௨৴Internal ELBΛར༻
Task Definitionͷํ ‣ ϩάͷసૹʹtd-agentΛ͏ ‣ ֤ίϯςφϩάϗετʹϚϯτ ‣ ϗετʹϚϯτ͞ΕͨϩάΛtd-agentͰసૹ ‣ logging
driver·ͩಋೖͯ͠ͳ͍ ‣ Internal Service(API)Ͱ͋ͬͯجຊNginxΛ௨͢ ‣ ΞΫηεϩάग़͢ͷָͩ͠
Task Definition(Service API)
Task Definition(WEB+API)
Task Definition(Job)
Task Definition(movie)
جຊతʹ͜ΕΒͷΈ߹Θͤ
cluster = serviceͷσϝϦοτ ‣ ϦιʔεޮతʹϕετͰͳ͍ ‣ Cluster : Service =
1:N ʹൺΔͱ ‣ Πϯελϯε૿͑Δʹ͋Δ ‣ ։ൃڥͰt2.microΛ༗ޮ׆༻͍ͯ͠Δ ‣ nanoΠϯελϯεщ ƅшƅщ)ŜŹŖƃ
cluster = service ʹͯ͠Δཧ༝ ‣ ࢹ֮తʹΘ͔Γ͍͢ʢϏΪφʔϑϨϯυϦʔʣ ‣ Service୯ҐͰIAM roleΛઃఆͰ͖ͳ͍ ‣
IAM RoleͰݫີͳݖݶ੍ޚΛ͢ΔʹΓͳ͍ ‣ Service୯ҐͰͷSecurity GroupʹະରԠ ‣ ͨͩ͠ɺELBલఏʹͯ͠͠·͑ղܾͰ͖Δ
Blue Green Deployment
2AutoScalingύλʔϯ ‣ BlueɺGreenܥ౷ͷClusterΛ࡞Δ ‣ ͦΕͧΕ͕AutoScalingGroupʹଐ͢Δ ‣ api1-blue, api1-green Έ͍ͨͳ ‣
AutoScalingGroup୯ҐͰELBΛΓସ͑Δ
None
2AutoScalingͷಛ ‣ ͱͯ҆શ ‣ DeployޙͷϩʔϧόοΫ༰қ ‣ DeployલͷStandbyܥ౷ͷΥʔϜΞοϓඞཁ ‣ 10લʹ͓͖͍ͬͯͨ ‣
ίετͷέΞ͕ॏཁ ‣ DeployޙɺStandbyʹͳͬͨܥ౷Λམͱ͢
Diet Docker Image
Πϝʔδখ͍͞΄Ͳྑ͍ ‣ docker build࣌ؒˣ ‣ CI࣌ؒˣ ‣ Registry͔ΒͷΠϝʔδμϯϩʔυ࣌ؒˣ ‣ AutoScaleͰ࡞͞ΕͨΠϯελϯε͕αʔϏεΠϯ͢Δ
·Ͱͷ࣌ؒ↓
docker hub ‣ hub.docker.com ‣ ଟ͘ͷެࣜΠϝʔδ ‣ αΠζΛ͗མͱͨ͠ͷ͔ΓͰͳ͍ ‣ 1GB͑ΔΠϝʔδσϒ
ෆཁͳͷআ͢Δ ‣ ෆཁͳϑΝΠϧΛݟམͱ͞ͳ͍ ‣ ϏϧυͷͨΊʹੜͨ͡࢈ۀഇغΛআ ‣ npm cache clear ‣
rm -rf ~/.grade ‣ apt-get clean ‣ Data VolumeΛ͏ʢϙʔλϏϦςΟམͪΔʣ
RUNͷճΛݮΒ͢ ‣ RUNͷ͚ͩΠϝʔδͷϨΠϠʔ͕ॏͳΓɺΠ ϝʔδ༰ྔ૿͑Δ ‣ && ͰνΣΠϯͯ͠ɺRUNͷճΛݮΒ͢ ‣ ͍docker buildͷ߹ɺ్தͰࣦഊ͢ΔͱRUN
ͷ಄͔ΒΓͳ͓͠ͳͷͰফ͢Δ
RUNͷճΛݮΒ͢ FROM ubuntu:15.10 RUN apt-get update RUN apt-get install -y
curl RUN apt-get apt-get clean FROM ubuntu:15.10 RUN apt-get update && \ apt-get install -y curl && \ apt-get apt-get clean
ܰྔΠϝʔδΛ͏ ‣ ͗མͱ͞ΕͨܰྔΠϝʔδΛ͏ ‣ ࠷ۙΦϑΟγϟϧͰslimΠϝʔδ͕͋Δͷ͋Δ ‣ e.g. Node, Go ‣
busyboxઈ͍ܰ
‣ ࠷ۙܰྔΠϝʔδ(slim)༻ҙ͞ ΕͯΔ ‣ Docker HubΛීஈ͔Β८ճ͓ͯ͠ ͜͏
‣ ٻΊΒΕΔϙʔλϏϦςΟˢ ‣ GoͰ͋Ε࣮ߦϑΝΠϧΛࡌͤΔ ͚ͩ ‣ busyboxͷதͰϏϧυΩπΠ ʢқߴʣ
ݮྔʹΑΔࢥΘ͵ฐ ‣ x509: failed to load system roots and no
roots provided ‣ ίϯςφ͔ΒHTTPS௨৴͕Ͱ͖ͳ͘ͳΔ ‣ apt-get install -y ca-certificates Ͱղܾ ‣ ֎෦πʔϧͷґଘʢΞϓϦ͔ΒͷΩοΫʣ
ϕʔεΠϝʔδΛͭ͘Δ ‣ apt updateɺapt-get install ͏Μ͵ΜΛऴΘΒͤͨ ͷ ‣ ຖͬͯͨΒCIͷ͕࣌ؒ૿͑Δʢdocker buildͷ
҆ఆʣ ‣ ΞϓϦଆͷDockerfileͰຊ࣭ͷॲཧʹઐ೦ͤ͞Δ
Local Development
ϩʔΧϧ։ൃͰ͏ͷ ‣ docker-machine + VirtualBox ‣ docker-compose
docker-machine ‣ VirtualBox, AWS, Azure, Digital Ocean্ʹ DockerϗετΛߏங͢Δ ‣ ͔͋ͨϩʔΧϧ্ʹDockerڥ͕͋Δ͔ͷΑ͏
ʹDockerͷૢ࡞͕Ͱ͖Δ
docker-machine ‣ docker-machine + VirtualBoxΛબ ‣ Vagrantࣺͯͨ ‣ DockerʹൺΔͱ͍ࣺͯίετˢ ‣
ΞϓϦέʔγϣϯɺϛυϧΣΞؚΊ͍ͯͭͰϩʔΧϧͰ֬ ೝͰ͖ΔΑ͏ʹ ‣ VirtualBoxͷϙʔτϑΥϫʔυར༻
ϚγϯϦιʔεඞཁ ‣ ϩʔΧϧϑϧDockerͩͱɺٻΊΒΕΔϚγϯεϖο Ϋߴ͘ͳΔ ‣ 16GBͳ͍ͱ݁ߏਏ͍ ‣ αʔόαΠυΤϯδχΞʹඞਢ ‣ ίϯςφ͍ͬͺ্͍͛ͯɺշదʹTwitter͢Δͷॏཁ
׳Εා͍
docker-compose ‣ Docker ToolboxͷҰ෦ʢݩʑfigʣ ‣ Dockerίϯςφ܈ͷߏΛYAMLͰཧ ‣ docker-compose up -d
Ͱىಈ
σʔλετΞDocker ‣ ϩʔΧϧͰσʔλετΞDockerίϯςφར༻ ‣ library/mysql, library/redis ‣ VagrantΑΓߴͰؾܰʹεΫϥοϓϏϧυ
DBϚΠάϨʔγϣϯॏཁ ‣ ڥἧͬͯσʔλෆඋ͋Δͱҙຯ͕ແ͍ ‣ FRESH! Ͱ goose ‣ https://bitbucket.org/liamstask/goose/ ‣
SQL͚ͩ͡Όͳ͘ɺGoͰϚΠάϨʔγϣϯॻ͚Δ
ecs-formation
ecs-formation ‣ https://github.com/stormcat24/ecs-formation ‣ docker-composeͷΑ͏ʹɺYAMLϑΝΠϧͰίϯςφ ͷߏΛཧ͢Δ ‣ ࣌ecs-cliͱ͔ແ͔ͬͨͷͰ࡞ͬͨ ‣ aws-sdk-goར༻
ecs-formation features ‣ Task Definitionsͷߋ৽ ‣ Clusterʹஔ͢ΔServiceͷߋ৽ ‣ Blue-Green Deploymentͷ࣮ߦ
Task Definitions (task/jawsug-api.yml) nginx: image: registry.jawsug.local:5000/jawsug/nginx:latest ports: - 80:80 environment:
SERVER_NAME: jawsug.example.com volumes: - /var/log/container/nginx:/var/log/nginx links: - api memory: 512 cpu_units: 512 essential: true
Services on cluster (service/jawsug-cluster.yml) api-service: task_definition: jawsug-api desired_count: 1
Blue Green Deployment (bluegreen/jawsug-cluster.yml) blue: cluster: jawsug-cluster-blue service: api-service autoscaling_group:
jawsug-cluster-blue green: cluster: jawsug-cluster-green service: api-service autoscaling_group: jawsug-cluster-green primary_elb: jawsug-api-primary standby_elb: jawsug-api-standby
ecs-formationͷӡ༻ ‣ ecs-formationઐ༻ͷϦϙδτϦΛͭ͘Δ ‣ Task, Service, BlueGreenͷఆٛͷYAML ‣ masterʹϚʔδ͞ΕΔͱTask Definitionߋ৽
‣ service updateHubot -> CircleCI͔Β
Update Task Definitions push merge PR webhook test update task
notification webhook ecs-formation repository Amazon ECS ※࠷৽ͷTask Definitionͷόʔδϣϯʹߋ৽͞ΕΔ
Update Service(Deploy) webhook test update service deploy webhook ecs-formation repository
Amazon ECS ※࠷৽ͷTask DefinitionͷόʔδϣϯΛར༻͠ɺServiceΛߋ৽͢Δ push deploy branch notification
Others
Others ‣ AMI ‣ Private Registry ‣ CircleCI + Docker
‣ Terraform ‣ Mackerel
EC2-Optimized AMI ‣ Current version 2015.09.b ‣ Amazon Linuxϕʔε ‣
Docker + ECS Agent ‣ ศར͚ͩͲࣾͰ໘ݟͯ͘Εͳͦ͞͏ͩͬͨ
Ubuntu ‣ FRESH!UbuntuΛ࠾༻ ‣ DockerͷΠϯετʔϧ → ηϧϑαʔϏε ‣ ECS AgentͷಋೖͱαʔϏεԽʢUpstartʣ
→ ηϧϑαʔϏε ‣ ੵۃతͳDockerͷΞοϓσʔτ → ਓப ‣ apt-get install docker-engine=1.9.0-0~trusty
Private Registry ‣ S3ΛόοΫΤϯυʹɺPrivate RegistryΛ࡞ΕΔ ‣ registry:2.2.0 ‣ konradkleine/docker-registry-frontend:v2
None
CircleCI + Docker ‣ ΞϓϦέʔγϣϯ1ϦϙδτϦʹ1Dockerfile ‣ nginxtd-agentͷΑ͏ͳϛυϧΣΞܥผ్ઐ ༻ϦϙδτϦ ‣ CircleCIͰmasterϏϧυ࣌ʹdocker
build + push
Build Docker Image push merge PR webhook test docker build
private registry docker push notification webhook
Terraform ‣ ΠϯϑϥߏஙͷͨΊͷΦʔέετϨʔγϣϯπʔ ϧ ‣ ଟ࠼ͳProviderΛఏڙ ‣ AWSͰͷΠϯϑϥߏஙʹར༻
Terraformཧͯ͠Δͷ ‣ EC2 ‣ Security Group ‣ Route53 (ΠϯλʔφϧυϝΠϯͷΈʣ ‣
ECS Cluster ‣ AutoScaling Groupͷىಈߏ
Terraformཧͷߟ͑ํ ‣ ӡ༻ʹΑͬͯঢ়ଶ͕มΘΔͷཧʹ͔ͳ͍ ‣ ELB ‣ AutoScaling Group ‣ εΫϥοϓʴϏϧυʹ͕͔͔࣌ؒΔͷ
‣ RDS ‣ ElastiCache ‣ EC2CloudinitͰߏ͢Δ ‣ Provisioning͠ͳ͍ɻ͍ࣺͯ
ΫϦςΟΧϧͳͷආ͚Δ ‣ Route53 ‣ Ͳ͔ͬͷϓϩδΣΫτͰɺϨϏϡʔ͕ܗ֚ԽˠηϧϑϚ ˠେࣄނ͕͋ͬͨΒ͍͠ ‣ roadworkerઐ༻ϦϙδτϦ༻ҙ͠ɺݫॏʹΫϩεϨ Ϗϡʔ͢Δ ‣
IAM
Terraformͷӡ༻ ‣ tfϑΝΠϧGHE্Ͱཧ ‣ tfstateS3্ʹอ࣋ ‣ શͯΛҰͭͷtfstateͰཧ͠ͳ͍ ‣ dev/shared/staging/production/load ͘Β͍
‣ CircleCI ‣ PRͰ terraform plan ͷࠩνΣοΫ ‣ planͰݫີͳνΣοΫͰ͖ͳ͍ͷͰͦΕͳΓʹ৺؟ඞཁ ‣ hubot -> CircleCIͰ terraform apply
Mackerel ‣ ࢹجຊతʹMackerel ‣ ݟ͍͢ɺ͖Ε͍ ‣ ࠷ۙDockerͷϝτϦΫε͕औΕΔΑ͏ʹͳͬͨ
Mackerel
Mackerel
Mackerel ˡίϯςφ୯Ґͷ ϝτϦΫε
࠷ޙʹॴײ ‣ ECSͪΖΜपลπʔϧἧ͖ͬͯͯɺDocker ຊ൪ӡ༻ͷෑډ͕େ͖͘Լ͕͍ͬͯΔ ‣ ΜͰΔͳΒͱΓ͋͑ͣࢼͯ͠ΈΑʁ
Thank you for listening