Share
Embed
Transcript
124 slides
-
Lobiͷαʔόมભ 1. 2010~2011 AWS (US) : 4(?) 2. 2011~2013 ࣗࣾαʔό
: 4 ~ 20 3. 2013.11~ AWS (Tokyo) : 20 ~ 100 -
Lobiͷαʔό EC2ͰՔಇ͍ͯ͠Δϗετͷछྨ͕ଟ͍ • app, sdk, stream, sdk-stream, db(3shard), transcode, log(aggregate,analyze),batch,
deploy... ࣮ݴޠ Perl, Node.js, Go ϛυϧΣΞ͍Ζ͍Ζ • Nginx, MySQL, Starlet, Fluentd, Norikra, memcached, HAProxy, gearman, twemproxy, MHA, dnsmasq... -
What's Consul www.consul.io - HashiCorp product • Service Discovery •
Health Checking • Key/Value Store • Multi Datacenter -
Agent ΫϥελͷnodeશͰಈ࡞͢Δdaemon Client mode or Server mode ͷͲͪΒ͔Ͱಈ࡞ ϢʔβʹDNS, HTTP
interfaceΛఏڙ͢Δ • Ϣʔβجຊతʹlocalhostͷagentͱ௨৴ • Agentಉ͕࢜RPCͰ௨৴(͋·Γҙࣝ͢Δඞཁ ͳ͍) GoͰॻ͔Ε͍ͯͯ1όΠφϦͰಈ࡞ (CLIಉҰ) -
Raft Raft is a protocol for implementing distributed consensus. ࢄڥͰͷ߹ҙΞϧΰϦζϜ(ϓϩτίϧ)
Leaderબग़ʹserver nodeͷաͷಉҙ͕ඞཁ ͳͷͰ࠷3 node͕ඞཁ -
Service / Node Discovery αʔϏεͷnodeͯ͢ͰΫϥελΛߏ͢Δ • nodeɺnode্Ͱఏڙ͢ΔαʔϏεͷൃݟ • DNS, HTTP
API ࢮ׆ࢹࣦഊͨ͠nodeԠ͔ΒࣗಈͰ֎ΕΔ -
Node Discovery consul members ͰΫϥελͷnodeΛҰཡ $ consul members Node Address
Status Type Build Protocol DC my-app-i-123456 192.168.1.12:8301 alive server 0.5.2 2 dc1 my-app-i-234567 192.168.1.23:8301 alive server 0.5.2 2 dc1 my-app-i-345678 192.168.1.34:8301 alive server 0.5.2 2 dc1 my-db-i-456789 192.168.1.45:8301 alive client 0.5.2 2 dc1 my-db-i-567890 192.168.1.56:8301 alive client 0.5.2 2 dc1 my-db-i-678901 192.168.1.67:8301 alive client 0.5.2 2 dc1 my-app-i-987654 192.168.1.99:8301 failed client 0.5.2 2 dc1 my-app-i-876543 192.168.1.87:8301 left client 0.5.2 2 dc1 (࣮ࡍ݁Ռͷॱ൪ෆఆ) -
Node Discovery Status=failed : agentͷࢮ׆ࢹʹࣦഊͨ͠node $ consul members -status failed
Node Address Status Type Build Protocol DC my-app-i-987654 192.168.1.99:8301 failed server 0.5.2 2 dc1 Status=left : ਖ਼ৗʹΫϥελ͔Βͨ͠node $ consul members -status left Node Address Status Type Build Protocol DC my-app-i-876543 192.168.1.87:8301 left server 0.5.2 2 dc1 -
Node Discovery via DNS interface consul agent (127.0.0.1:8600) ʹ͍߹ΘͤΔ $
dig @127.0.0.1 -p 8600 my-app-i-123456.node.consul ;; QUESTION SECTION: ;my-app-i-123456.node.consul. IN A ;; ANSWER SECTION: my-app-i-123456.node.consul. 0 IN A 192.168.1.12 -
Node Discovery via DNS interface Status=failed : DNSͰΞυϨε͕Ҿ͚Δ • Ұ࣌తʹࣄނͰ͍ͯ͠ΔՄೳੑ͕͋Δ
Status=left : DNSͰΞυϨε͕Ҿ͚ͳ͘ͳΔ • .consul υϝΠϯͷ໊લղܾΛ consul agent ʹ ͛Δ͜ͱͰ෦DNSͱͯ͠ར༻Ͱ͖Δ • υϝΠϯ໊ઃఆͰมߋՄೳ -
Service Discovery of App via DNS interface consul agent (127.0.0.1:8600)
ʹ͍߹ΘͤΔ $ dig @127.0.0.1 -p 8600 app.service.consul ;; QUESTION SECTION: ;app.service.consul. IN A ;; ANSWER SECTION: app.service.consul. 0 IN A 192.168.1.12 app.service.consul. 0 IN A 192.168.1.23 app.service.consul. 0 IN A 192.168.1.34 -
Service Discovery via DNS interface Answerͷॱ൪ϥϯμϜ (≒ Round Robin) UDPͰͷ͍߹ΘͤͰର͕4ΞυϨεҎ্
͋Δ߹ɺ3ΞυϨεͷΈฦΔ • TCPͰͯ͢ฦΔ TTL ઃఆՄೳ (default 0) -
Service Discovery of App via HTTP API http://127.0.0.1:8500 ʹΞΫηε $
curl http://127.0.0.1:8500/v1/catalog/service/app [ { "Node": "my-app-i-123456", "Address": "192.168.1.12", "ServiceID": "app", "ServiceName": "app", "ServicePort": 3000, ... }, { "Node": "my-app-i-234567", "Address": "192.168.1.23", ... } { "Node": "my-app-i-345678", "Address": "192.168.1.34", ... } ] -
Service Difinision of DB (master) my-db-i-456789:/etc/consul.d/db.json { "service": { "name":
"db", "port": 3306, "tags": ["master"] } } -
Service Difinision of DB (slave) my-db-i-567890,678901:/etc/consul.d/db.json { "service": { "name":
"db", "port": 3306, "tags": ["slave"] } } -
Service Discovery of DB (master/slave) {tag}.{service}.service.consul Ͱ໊લղܾ $ dig @127.0.0.1
-p 8600 master.db.service.consul master.db.service.consul. 0 IN A 192.168.1.45 $ dig @127.0.0.1 -p 8600 slave.db.service.consul slave.db.service.consul. 0 IN A 192.168.1.67 slave.db.service.consul. 0 IN A 192.168.1.56 $ dig @127.0.0.1 -p 8600 db.service.consul db.service.consul. 0 IN A 192.168.1.56 db.service.consul. 0 IN A 192.168.1.45 db.service.consul. 0 IN A 192.168.1.67 -
External Service nodeʹؔ࿈͠ͳ͍ɺ֎෦DNSͰఆٛ͞Ε͍ͯΔ ໊લIPΞυϨεαʔϏεͱͯ͠ఆٛͰ͖Δ $ curl -X PUT -d '{
"Node":"rds", "Address":"my-rds.xxxxx.ap-northeast-1.rds.amazonaws.com", "Service":{"Service": "rds"} }' http://127.0.0.1:8500/v1/catalog/register $ dig @127.0.0.1 -p 8600 rds.service.consul. ;; ANSWER SECTION: rds.service.consul. 0 IN CNAME my-rds.xxxxx.ap-northeast-1.rds.amazonaws.com. my-rds.xxxxx.ap-northeast-1.rds.amazonaws.com. 10 IN A 192.168.1.100 -
External Service DNS໊Λొ͢Δ߹ɺconsulͷઃఆʹ recursors (֎෦ͷ໊લղܾΛ͢ΔDNSαʔόͷ ΞυϨε) Λ͓ͯ͘͠ • ઃఆ͠ͳ͍ͱ CNAME
ͷΈ͕ฦΔ consul agent֎෦ͷ໊લղܾ݁ՌΛcache͠ͳ ͍ -
Health Checking by script ϢʔβఆٛͷϔϧενΣοΫίϚϯυΛ࣮ߦ exit codeͰঢ়ଶΛ௨ (Nagios/Sensuޓ) • 0
: success • 1 : warning • 2 : fail failͷ߹DNS, HTTPͷԠ͔Β֎ΕΔ -
Health Checking by HTTP consul agent͕HTTPͰΞΫηε • HTTP 2xx :
success • HTTP 429 : warning • ͦΕҎ֎ : fail -
Key/Value Store ҙͷΛग़͠ೖΕͰ͖ΔKVS $ curl -XPUT -d 'test' 'http://127.0.0.1:8500/v1/kv/web/key1' true
$ curl http://127.0.0.1:8500/v1/kv/web/key1 [{ "CreateIndex": 112, "ModifyIndex": 112, "LockIndex": 0, "Key": "web/key1", "Flags": 0, "Value": "dGVzdA==" }] -
Key/Value Store URLҾͰϝλσʔλ (flags) Λอ࣋Ͱ͖Δ 64bit int, ༻్Ϣʔβͷҙ $ curl
-XPUT -d 'test' 'http://127.0.0.1:8500/v1/kv/web/key1?flags=123' true $ curl 'http://127.0.0.1:8500/v1/kv/web/key1' [{ ... "Key": "web/key1", "Flags": 123, // <------- ͜Ε "Value": "dGVzdA==" }] -
Key/Value Store ͋Δ֊ͷԼͷΛ࠶ؼతʹऔΓ͍ͨ߹ Ҿ recurse $ curl -s "http://127.0.0.1:8500/v1/kv/web/?recurse" [
{"CreateIndex":112,"ModifyIndex":115,"LockIndex":0, "Key":"web/key1","Flags":123,"Value":"dGVzdA=="}, {"CreateIndex":122,"ModifyIndex":122,"LockIndex":0, "Key":"web/key2","Flags":0,"Value":"dGVzdDI="}, {"CreateIndex":124,"ModifyIndex":124,"LockIndex":0, "Key":"web/test/1","Flags":0,"Value":"dGVzdDM="} ] όοΫΞοϓʹར༻Մೳ -
Key/Value Store Benchmark GET $ wrk -c 10 -d 10
-t 2 http://127.0.0.1:8500/v1/kv/web/key1 Server(Leader): 41,832 Requests/sec Server/Client(Follower): 17,281 Server(Follower) stale mode: 37,013 Client(Follower) stale mode: 16,938 Consul v0.5.2 on EC2 c4.2xlarge, GOMAXPROCS=4 -
Key/Value Store Benchmark PUT $ wrk -c 10 -d 10
-t 2 -s put.lua http://127.0.0.1:8500/v1/kv/web/key1 -- put.lua wrk.method = "PUT" wrk.body = "test" wrk.headers["Content-Type"] = "application/x-www-form-urlencoded" Server/Client(Follower): 427.56 Requests/sec -
Key/Value Store ৗʹ 127.0.0.1:8500 Λ͑Α͍ͷͰָ ͔ͳΓߴͳͷͰGETԕྀͳ͑͘Δ σϑΥϧτͰͯ͢ͷ͍߹ΘͤΛ Leader node͕ॲཧ͢Δ •
stale modeʹ͢ΔͱLeaderҎ֎ͷServerԠ Ͱ͖Δ (Ұ؏ੑʹ͍ͭͯޙड़) -
෦DNSͱͯ͠ConsulΛ͏ node,serviceͷ໊લղܾΛConsul Agent͚Δ ! resolv.confͰϙʔτࢦఆͰ͖ͳ͍ ! Port 53ಛݖ͕ඞཁ → ͦͷͨΊ͚ͩʹAgentΛrootͰಈ͔͢ͷ
! Agent࠶ؼ͍߹Θͤ(ҰԠ)Ͱ͖Δ͕cacheػ ೳ͕ͳ͍ ! → dnsmasq, bindͳͲ͔Β.consulυϝΠϯͷΈ forward ! -
෦DNSͱͯ͠ConsulΛ͏ dnsmasqΛશͰىಈ .consul υϝΠϯͷ໊લղܾconsul agent 127.0.0.1:53 Λdnsmasq͕Listen͢Δ # dnsmasq.conf server=/consul/127.0.0.1#8600
bind-interfaces listen-address=127.0.0.1 -
෦DNSͱͯ͠ConsulΛ͏ resolv.conf Ͱ (node|service).consul Λݕࡧυ ϝΠϯʹࢦఆ → node໊ɺservice໊͚ͩͰଓͰ͖Δ # /etc/resolv.conf
search node.consul service.consul nameserver 127.0.0.1 # dnsmasq nameserver 172.16.0.2 # VPC resolver nameserver 172.16.0.254 # Unbound on EC2 -
ServerʹඞཁͳϦιʔε • CPU: 2CPUͰे • GOMAXPROCS=2 Ҏ্Ͱಈ࡞ͤ͞ΔͷΛڧ͘ਪ • Memory:
20MBʙ • Disk: 2MBʙ Memory, DiskKVͷར༻ঢ়گ࣍ୈ -
Serverʹઐ༻ϗετ͕ඞཁʁ consul agentࣗମͦΕ΄ͲϦιʔεΛ ༻͠ͳ͍ͨΊɺಉډՄೳ͕ͩ… Disk IO͕ߴෛՙͳ߹ʹRaftͷHeartbeat͕ ࣦഊ͍͢͠ • Timeout 500ms
• Heartbeatʹࣦഊ͢ΔͱLeaderબग़͕ߦΘΕΔ • ௨ৗ2,3ඵͰબग़ྃ͢Δ • ͦͷؒॻ͖ࠐΈॲཧ͕Ͱ͖ͳ͍ -
Daemonize consul agentࣗDeamonಈ࡞Ϟʔυ͕ͳ͍ • Daemontools • RPM & init script
• github.com/tomhillable/consul-rpm • Systemd ͲΕͰ͓ΈͰ -
Bootstrapping consul agentىಈޙɺΫϥελʹjoin͢Δඞཁ ͕͋Δ 1. ίϚϯυͰ consul joinΛ࣮ߦ 2. ઃఆϑΝΠϧͰ
start_join Λࢦఆ 3. Atlas࿈ܞ -
2. start_join consul.confʹαʔόͷΞυϨεΛࢦఆ͓ͯ͘͠ { "start_join": [ "192.168.1.11", "192.168.1.12", "192.168.1.13"] }
consul serverʹݻఆIPΞυϨεΛৼΔ͔ɺผͷ DNSͰղܾͰ͖ΔΑ͏ʹ͓ͯ͘͠ -
3. Atlas࿈ܞ Atlas - atlas.hashicorp.com Vagrant Packer Terraform ConsulΛ౷߹͢ΔαʔϏε $
consul agent ... \ -atlas=ATLAS_USERNAME/infrastructure \ -atlas-join \ -atlas-token="YOUR_ATLAS_TOKEN" \ ! ServerͷΞυϨεΛཧ͢Δඞཁ͕ͳ͍ͷͰָ " 11nodeҎ্$40/node -
ߴՄ༻ੑͷͨΊʹ ServerʹΑΓಉ࣌ʹো Λىͯ͜͠ͳ͍node ͕มΘΔ • 3 node → 1 •
5 node → 2 3 nodeߏ࣌ɺ2མͪͯ Γ1ʹͳͬͯ͠·͏ͱLeader ͕બग़Ͱ͖ͳ͍ ࣌ؒΓ͢ϝϯςφϯε ࣌ʹҰ࣌తʹServer nodeΛ ૿͢ख -
nodeো࣌ͷӨڹ ! LeaderͰͳ͍ → " ଞnodeʹӨڹͳ͠ ! Leader → "
Leader࠶બग़ σϑΥϧτͰͯ͢ͷಡΈॻ͖ΛLeader͕ॲཧ (ڧҰ؏ੑ) Leader͕ܾ·Δ·ͰΞΫηεෆೳ (DNS, HTTP) -
Stale mode (DNS) Leader࠶બग़௨ৗ2ʙ3ඵͰྃ ͦͷؒDNSͰNode, Service໊ղܾΛ͍ͨ͠ʁ → Stale mode :
Leaderະબग़ͰԠՄೳ "dns_config":{ "allow_stale": true, // default false "max_stale": "10s" // default 5s } ݁Ռݹ͍Մೳੑ͕͋Δ(݁Ռ߹ੑ) -
DNS TTL defaultTTL 0 → cache͞Εͳ͍ node, serviceผʹTTLΛઃఆՄೳ DNS cacheΛલஈʹஔͯ͠cacheͰ͖Δ
"dns_config":{ "node_ttl": "60s", "service_ttl": { "*": "15s" } } -
Stale mode (HTTP API) HTTP APIͰstale modeʹ͢Δ߹Ҿ stale $ curl
"http://127.0.0.1:8500/v1/kv/web/key1?stale" staleҾͳ͠ͰLeaderબग़தʹΞΫηε → 500 Internal Server Error -
Agentࣗମͷࢹ Agent processࣗମͷࢹผ్ • processࢹ(consul agent) • TCP/UDP 8600 (DNS)
• TCP 8500 (HTTP) • http://127.0.0.1:8500/v1/status/leader ༰มߋݕ • Leader Lost→࠶બग़ͰมΘΔ -
ElasticTranscoder ! Managed ServiceͳͷͰཧָ͕ ! มೳྗউखʹεέʔϧ ! ͪΐͬͱ͓ߴ͍… • SD
$0.017/min • HD $0.034/min ߘ͋ͨΓ4ύλʔϯ ฏۉ2 = $0.204 ≒ 25ԁ -
EC2 Spot InstanceͰಈըม ! ཧ͕໘ ! উखʹεέʔϧͯ͘͠Εͳ͍ ! ElasticTranscoderΑΓѹతʹ҆Ձ •
ElasticTranscoder = $0.204/ߘ • EC2 Spot cc2.8xlarge(32core) = $0.45/hour -
Spot InstanceͰΦʔτεέʔϧ • εέʔϧΞτ: CPUෛՙͰͳ͘JobͷྔͰ • Job͕ཷ·Βͳ͍ݶΓCPU 100% ! •
εέʔϧΠϯ: CPU idle͕શମͷ25%Ҏ্ -
1. ࣗಈͰuniqueͳϗετ໊Λ͚Δ ConsulͷͨΊʹҰҙͳhostname͕ඞཁ Cloud-InitͰىಈ͢ΔϗετͷλΠϓΛઃఆ #cloud-config runcmd: - [sh, -c, 'echo
"HOSTNAME_PREFIX=transcode" > /etc/sysconfig/hostname-prefix'] rc.localͰಡΈࠐΉ # /etc/rc.local if [ -f /etc/sysconfig/hostname-prefix ]; then . /etc/sysconfig/hostname-prefix fi -
1. ࣗಈͰuniqueͳϗετ໊Λ͚Δ hostnameΛ prefix + InstanceID EC2 Name tag ༩
# /etc/rc.local instance_id=$(curl -s 169.254.169.254/latest/meta-data/instance-id) new_hostname="${HOSTNAME_PREFIX}-$instance_id" hostname $new_hostname aws ec2 create-tags \ --resources $instance_id \ --tags "Key=Name,Value=$new_hostname" -
3. ChefʹΑΔϓϩϏδϣχϯά Consul KV͔ΒϗετλΠϓ͝ͱͷJSONΛऔಘ͠ ࣮ͯߦ JSON=$(curl -s "localhost:8500/v1/kv/nodes/bootup/${HOSTNAME_PREFIX}.json?raw") echo "$JSON"
> /tmp/chef-bootup.json chef-client -j /tmp/chef-bootup.json ॳճChef࣮ߦ࣌ʹChef-Serverʹొ͞ΕΔ -
Why Stretcher? Archer(rsync) ʹΑΔதԝϗετ͔Βͷdeploy ! pushͰΦʔτεέʔϧʹରԠͰ͖ͳ͍ ! ֤ϗετ͔ΒrsyncͰpull? → buildதʹrsync͞ΕͨΒ…
! ֤ϗετ͔Βgit pull? → grunt, GoͳͲͷbuildੜΛೖΕͨ͘ͳ͍ ! ͕ଟ͍ͱssh+rsyncgit pullπϥ͍ ! AMI࡞Γ͠&ೖΕସ͑ͯͳ͍ -
Consul event ֤nodeʹGossip ProtocolͰΠϕϯτΛ ૹ৴͢ΔΈ $ consul event -name EVENT_NAME
[-node REGEX] PAYLOAD Event ID: 3b1f3199-6e69-4b82-4812-b35058864fdd ࢦఆͨ͠Πϕϯτ໊Ͱ (ਖ਼نදݱʹϚον͢Δnodeʹ) payloadΛૹ৴ -
Consul watch ࢦఆͨ͠ΠϕϯτΛड৴ͨ͠ΒίϚϯυΛ ࣮ߦ͢ΔΈ $ consul watch -type event -name
EVENT_NAME COMMAND payloadඪ४ೖྗ͔ΒJSONͰ͞ΕΔ [{ "ID": "3b1f3199-6e69-4b82-4812-b35058864fdd", "Name": "test", "Payload": "TXkgcGF5bG9hZA==", ... }] -
Deployment process 1. ΞϓϦέʔγϣϯΛbuildͯ͠tar.gzʹ͢Δ ґଘcpan moduleͳͲͯ͢ݻΊΔ 2. खॱॻ(manifest)Λॻ͘ 3. tar.gz,
manifestΛS3(or httpd)ʹ্͛Δ 4. consul event Ͱ manifest URLΛ௨ consul event -name deploy s3://... ✄---------- ͜͜·ͰstretcherͰͳ͍ ---------✄ -
✄------------ stretcher͔͜͜Β ------------✄ consul watch -type event -name deploy stretcher
1. event͔Βmanifest URLΛऔಘ 2. tar.gzΛऔಘͯ͠TMPDIRʹల։ 3. rsync -av --deleteͰߋ৽ 4. command࣮ߦ • ΞϓϦέʔγϣϯ࠶ىಈͳͲ -
Manifest src: s3://example.com/app.tar.gz checksum: e0840daaa97cd2cf2175f9e5d133ffb3324a2b93 dest: /home/stretcher/app commands: pre: -
echo 'staring deploy' post: - echo 'deploy done' success: - cat >> /path/to/success.log failure: - cat >> /path/to/failure.log excludes: - "*.pid" - "*.socket" -
LobiͰͷdeploy tar.gz 200MB ల։͢Δͱ400MB • CPAN modules 110MB • node_modules
10MB × 5 • Go app binaries 8MB × 5 • Static files (S3ʹஔ͖͍ͨ) -
LobiͰͷdeploy 1. ‐ Push to production branch 2. ! Build
(1 min~) carton install, grunt, npm install, go build ... 3. " Pack tar.gz & Upload (1 min) 4. # Deploy by Startecher (10~30 sec) -
LobiͰͷdeploy consul event ૹ৴͔Β10ʙ20ඵͰྃ ! 2015/08/05 14:58:30 Starting up stretcher
agent 2015/08/05 14:58:30 Waiting for consul events from STDIN... 2015/08/05 14:58:30 Executing manifest: s3://... 2015/08/05 14:58:33 Extract archive: /dev/shm/stretcher539962129 to /dev/shm/stretcher_src648982332 2015/08/05 14:58:36 rsync [-av --delete --exclude-from /dev/shm/stretcher_src648982332/conf/rsync_exclude.web /dev/shm/stretcher_src648982332/ /home/xxx/web/] 2015/08/05 14:58:36 sending incremental file list ... sent 787780 bytes received 5230 bytes 1586020.00 bytes/sec total size is 359702435 speedup is 453.59 2015/08/05 14:58:36 invoking command: /home/xxx/web/refresh_services.sh 2015/08/05 14:58:41 success. 2015/08/05 14:58:41 Deploy manifest succeeded. Rollback͍ͨ͠ˠલͷmanifestΛeventૹ৴ →10secͰΔ -
IRC / Slackʹ௨ʁ ࣾ༻ nopaste-cli command $ nopaste-cli -channel "lobi"
-summary "Deploy done!" < deploy.log nopasteʹPOSTͱಉ࣌ʹURL͕௨͞ΕΔ 100͋Δͱ௨͕……!!!!!!!!×100 Ͳ͔͜ͰҰཡͯ͠ݟ͍ͨʂ -
σʔλొ Consul HTTP APIͰૹΔ $ curl -X PUT -d "message"
\ '127.0.0.1:8500/v1/kv/dashboard/example/myhostname?flags=1422607461000' -
keyߏ /v1/kv/dashboard/{category}/{nodename}? flags=({unixtime} * 1000 + {status}) • category: chef,
serverspec, deploy... • nodename: Consulͷnode໊ • flags: unixtime * 1000 + status • status: 0=Success 1=Warning 2=Danger 3=Info -
ը໘ͷଈ࣌ߋ৽ ϒϩοΩϯάΫΤϦΛ͏ consul.io/docs/agent/http.html $ curl -i 127.0.0.1:8500/v1/kv/dashboard/chef/myhost?recurese HTTP/1.1 200 OK
Content-Type: application/json X-Consul-Index: 261975 [{"CreateIndex":261891,"ModifyIndex":261975,"LockIndex":0, "Key":"dashboard/chef/myhost","Flags":1422602855000,"Value":"..... -
Blocking query Ϩεϙϯεϔομͷ X-Consul-Index Λ ࣍ͷϦΫΤετͷҾʹࢦఆ $ curl -i 127.0.0.1:8500/v1/kv/dashboard/chef/myhost
HTTP/1.1 200 OK X-Consul-Index: 261975 ... $ curl 127.0.0.1:8500/v1/kv/dashboard/chef/myhost?index=261975 ৽͍͠σʔλ͕ൃੜ͢Δ·ͰϨεϙϯε͕Ԇ ͍ΘΏΔ Long pooling -
Blocking queryͷ׆༻ Consul Template github.com/hashicorp/consul-template KV, node, serviceͷঢ়ଶมԽΛଈө Template→fileߋ৽ˠcommand࣮ߦ $
consul-template \ -consul 127.0.0.1:8500 \ -template "/tmp/template.ctmpl:/var/www/nginx.conf:service nginx restart" -
consul-kv-dashboard͕Δ͜ͱ(2) Consul HTTP API ͷ reverse proxy • /api/... →
127.0.0.1:8500/v1/kv/dashboard/... • ϨεϙϯεͷJSONΛܗ {"Flags":1422608524001} ! {"timestamp":"2015-01-30 18:02:04 +0900","status":"warning"} • /v1/catalog/nodes Λblocking queryͰࢹ ଘࡏ͢ΔnodeͷσʔλͷΈϑΟϧλ -
consul-kv-dashboard trigger $ consul-kv-dashboard -trigger COMMAND ΧςΰϦຖʹঢ়ଶ(success→warningͳͲ)͕ มԽͨ͠ΒίϚϯυ࣮ߦՄೳ JSON͕ඪ४ೖྗʹ͞ΕΔ {
"category":"testing", "node":"web01", "address":"192.168.1.10", "timestamp":"2015-01-21 11:22:33 +0900", "status":"danger", "key":"","data":"failure!!" } -
ىಈޙʹ࠷৽൛Λdeploy deploy࣌ɺKVʹ࠷৽ͷmanifest URLΛอଘ 1. consul join 2. stretcherىಈ 3. ࠷৽ͷmanifest
URLΛKV͔Βऔಘ 4. ࣗࣗʹ event (manifest URL) Λૹ৴ 5. deploy ! -
ΦʔτεέʔϧͰͷҙ ! AMIʹ͍ͬͯΔݹ͍ΞϓϦ͕ىಈ " ࠷৽ͷ deploy IDͰͳ͍߹ىಈ͠ͳ͍ deploy࣌: unique ͳ
ID Λൃߦ • ϑΝΠϧʹॻ͍ͯ tar ʹೖΕΔ • KV ʹೖΕΔ ىಈ࣌:ϩʔΧϧϑΝΠϧͷ ID ͱ KV Λൺֱ • ҟͳ͍ͬͯͨΒ sleep 10 && exit → restart -
bash-completionͰsshͷϗετ໊ิ ~/.bash_profile _known_hosts_real() { local members=$(consul members -status=alive | awk
'!/Node/{printf("%s ", $1)}') COMPREPLY=( $( \ compgen -W "$members" \ ${COMP_WORDS[COMP_CWORD]} \ ) ) return 0 } ! ݱࡏaliveͳϗετͷΈ͕ग़ͯ͘Δʂ -
Questions? • Architecture, Service Discovery Health Checking, Key/Value Store •
෦DNS • ຊ൪ڥӡ༻ / ߴՄ༻ੑ • Φʔτεέʔϧ • Stretcher • Consul KV Dashboard