Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Monitoring Kubernetes
Search
Shinji Tanaka
December 13, 2014
Technology
1
6.4k
Monitoring Kubernetes
gcp ja night #29
12 Dec 2014
Shinji Tanaka
December 13, 2014
Tweet
Share
More Decks by Shinji Tanaka
See All by Shinji Tanaka
Mackerel loves Machine Learnings
stanaka
0
3k
How to Contribute Go
stanaka
4
18k
Introduction to Mackerel
stanaka
0
1k
Upcoming features in 2016, Mackerel
stanaka
0
3.4k
Mackerel's first year and onward
stanaka
0
3k
Mackerel's first year and onward (in English)
stanaka
0
400
Blue-Green Deployment with ECS and monitoring
stanaka
0
620
Mackerel Meetup #4 (in Eng)
stanaka
0
520
Mackerel Meetup #4
stanaka
0
3.2k
Other Decks in Technology
See All in Technology
テストを実施する前に考えるべきテストの話 / Thinking About Testing Before You Test
nihonbuson
PRO
13
2k
Oracle Database オプティマイザ・ヒントの活用
oracle4engineer
PRO
1
140
大事なのは、AIの精度だけじゃない!〜1円のズレも許されない経理領域とAI〜
jun_nemoto
11
5.1k
toittaにOpenTelemetryを導入した話 / Mackerel APM リリースパーティ
cohalz
1
490
SmartHRの複数のチームにおけるMCPサーバーの活用事例と課題
yukisnow1823
2
1.1k
LT:組込み屋さんのオシロが壊れた!
windy_pon
0
350
大手企業のAIツール導入の壁を越えて:サイバーエージェントのCursor活用戦略
gunta
9
1.4k
コードの考古学 〜労務システムから発掘した成長の糧〜
kenta_smarthr
1
1.1k
エンジニアが組織に馴染むために勉強会を主催してチームの壁を越える
ohmori_yusuke
2
120
Data Hubグループ 紹介資料
sansan33
PRO
0
1.7k
What's Next in OpenShift Q2 CY2025
redhatlivestreaming
1
790
大規模PaaSにおける監視基盤の構築と効率化の道のり
lycorptech_jp
PRO
0
180
Featured
See All Featured
Designing for Performance
lara
608
69k
Mobile First: as difficult as doing things right
swwweet
223
9.6k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3k
Building Applications with DynamoDB
mza
95
6.4k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
How STYLIGHT went responsive
nonsquared
100
5.6k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.3k
Embracing the Ebb and Flow
colly
85
4.7k
Agile that works and the tools we love
rasmusluckow
329
21k
Documentation Writing (for coders)
carmenintech
71
4.8k
Six Lessons from altMBA
skipperchong
28
3.8k
Transcript
Monitoring Kubernetes gcp ja night #29 Hatena Co., Ltd. @stanaka
! 2014/12/12
[PR] Mackerel https://mackerel.io/
Kubernetes • Docker ΛΫϥελཧ! • ͍Ζ͍ΖͳڥͰಈ͔͍ͨ͠ • MackerelͰϦιʔεཧ͍ͨ͠
Kubernetes Monitoring ʹؔ࿈͢Δཁૉ • kubelet • cAdvisor • heapster
kubelet • ֤ϊʔυ(minion)Ͱಈ࡞ • kubernetes masterͱڠௐಈ࡞
cAdvisor • Dockerͷ౷ܭใΛऔಘ • ϗετશମͱίϯςφຖ • DockerҎ֎ͷίϯςφରԠ HPPHMFDBEWJTPS
heapster • Ϧιʔεใऩूπʔϧ • kubarnetesઐ༻ • InfluxDBΠϯϙʔτ (PPHMF$MPVE1MBUGPSNIFBQTUFS
kubarnetesඪ४ϞχλϦϯά
kubarnetesඪ४ϞχλϦϯά • heapster + cAdvisor → InfuxDB & Grafana
• Ϋϥελʹ͋ΔͷͰɺΫϥελΛ མ͢ͱམͪΔ
Minionͷ౷ܭใऔಘ • http://#{ipaddr}/stats/ • kubeletͷAPI • ཪͰcAdvisor͕ಈ͘ $ curl
http://10.240.153.49:10250/stats/ | jq 'keys' [ "name", “spec", # εϖοΫ “stats", # ౷ܭใ 1ඵ͝ͱ 60ඵؒ "subcontainers" ]
statsͷத $ curl http://10.240.56.195:10250/stats/90559c9f-‐8123-‐11e4-‐ a0ec-‐42010af08f91/php-‐redis | jq ".stats |
.[] | .timestamp" "2014-‐12-‐12T02:41:55.52640117Z" "2014-‐12-‐12T02:41:56.526265405Z" "2014-‐12-‐12T02:41:57.52632129Z" "2014-‐12-‐12T02:41:58.526385146Z" "2014-‐12-‐12T02:41:59.526285874Z" "2014-‐12-‐12T02:42:00.526267704Z" "2014-‐12-‐12T02:42:01.526227391Z" "2014-‐12-‐12T02:42:02.526312936Z" "2014-‐12-‐12T02:42:03.526225182Z" “2014-‐12-‐12T02:42:04.526287033Z" … (ҎԼɺ60ඵ)
statsͷத #2 # curl http://10.240.153.49:10250/stats/ | jq '.stats
| .[0] | keys' [ "cpu", "diskio", "filesystem", "memory", "network", "timestamp" ]
statsͷதৄࡉ UJNFTUBNQ DQV NFNPSZ OFUXPSL \SY UY^@\CZUFT QBDLFUT FSSPST
ESPQQFE^ pMFTZTUFN EFWJDF<> DBQBDJUZ VTBHF JP@UJNF EJTLJP JP@NFSHFE JP@RVFVFE JP@TFSWJDF@CZUFT NBKPS NJOPS TUBUT "TZOD 3FBE 4ZOD 5PUBM 8SJUF JP@TFSWJDF@UJNF JP@TFSWJDFE JP@UJNF JP@XBJU@UJNF TFDUPST
ϞχλϦϯάख๏ • Ϋϥελ্Ͱಈ͍͍ͯΔͷΛѲ • kubecfg list pods • kubecfg get
pods/#{ID} • Munion্ͷίϯςφͷঢ়ଶऔಘ • /stats/#{ID}/#{container}
kubecfg list pods • cluster/kubecfg.sh -json list pods \
JUFNT< \ JEGDFBFDBGG IPTULVCFSOFUFTNJOJPODCVPZBOU[PEJBDJOUFSOBM EFTJSFE4UBUF\ NBOJGFTU\ DPOUBJOFST< \ JNBHFCSFOEBOCVSOTQIQSFEJT OBNFQIQSFEJT
kubecfg get pods • cluster/kubecfg.sh -json get pods/ 73f777c6-8123-11e4-a0ec-42010af08f91 •
Ϩεϙϯε list pods ͱಉ͡
Munion্ͷίϯςφͷঢ়ଶऔಘ • curl http://10.240.56.195:10250/ stats/90559c9f-8123-11e4- a0ec-42010af08f91/php-redis | jq ".stats |
.[] | .timestamp” • Ϩεϙϯε/stats/ͱಉ͡
Φʔτεέʔϧ͍ͤͨ͞ • Ϧιʔεใऔಘ • αΠζมߋஅ • αΠζมߋ࣮ߦ(ඞཁ͕͋Ε)
podsͷಈతͳαΠζมߋ • resizeͰมߋ cluster/kubecfg.sh resize frontendController 4 $ cluster/kubecfg.sh
list replicationControllers Name Image(s) Selector Replicas -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ frontendController brendanburns/php-‐redis name=frontend 4
Kubenetesҹ • ͍Ζ͍Ζศརʹ͑ͦ͏Ͱເ͋Δ • ϓϩμΫγϣϯʹಥͬࠐΉͷ·ͩ… • དྷՆ͙Β͍ʹ͜ͳΕ͖ͯͦ͏?