20 Jan 2017 / This is Mercari, This is a SRE - Mercari DAY

Presentation at #mercariday

Transcript

1. This is Mercari, This is an SRE Mercari DAY 2017 @AcademyHills Kenichi Sasaki
2. Introduce myself • Kenichi Sasaki (@siroken3) • Mercari, Inc (JOIN at 2014/7)
 Site Reliability Engineering (SRE) Team • αʔόϓϩϏδϣχϯάɾσϓϩΠͷ੔උ • ։ൃ؀ڥͳͲͷ੔උ
3. Agenda • Site Reliability Engineering (SRE) ͱ͸ • MercariͱMercariͷSRE • Mercari SREͷࣄྫ • ·ͱΊ
4. SRE ͱ͸
5. 4JUF
   3FMJBCJMJUZ
   &OHJOFFSJOH w (PPHMF
   ͕ఏএ
   w (PPHMF
   ͷ༷ʑͳϓϩμΫτɾ αʔϏεΛԣஅͯ͠ɺαΠτ ͷ৴པੑΛ޲্ͤ͞Δ
   4PGUXBSF
   &OHJOFFS5FBN
6. SRE ͷ࠾༻(US) Facebook, Twitter, Apple, eBay, Microsoft.. 3,662
7. SRE in JP • 2015/11 Mercari Tech Blog ʹͯ঺հͯ͠Ҏ߱஫ ໨͕ू·Δ • “ΠϯϑϥνʔϜվΊ Site Reliability Engineering (SRE) νʔϜʹͳΓ·ͨ͠”
 http://tech.mercari.com/entry/2015/11/18/153421 • 2016/06 SRE Tech Talk#1 • https://connpass.com/event/34825/
8. SRE Tech Talk#2! • https://eventdots.jp/event/609456 • 2017/01/23 12:00 క੾
9. SREͷ࠾༻(JP) 40!
10. Mercari & Mercari SRE
11. Infrastructure UK ४උத JP US ͘͞ΒΠϯλʔωοτ
 ੴङDC AWS GCP
12. Architecture US nginx nginx nginx ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. Client Multimedia Corporate data center Traditional server Mobile Client IAM Add-on Example: IAM Add-on ence ) Assignment/ Task Requester Workers DNS-RR App App App App App App MySQL MySQL memcached memcached nginx nginx nginx ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia C d Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific DNS-RR App App App App App App MySQL MySQL memcached memcached EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 JP Ϋϥ΢υͰ΋
    ಉ͡ߏ੒ util util cloud cloud util util EC2 EC2
13. Mercari SRE Team
14. Mercari SRE • 2015/11 ͔Β SRE ʹมߋ
 ͦΕ·Ͱ͸ΠϯϑϥνʔϜ • ϝϯόʔ͸5ਓ • શһ͕౦ژΦϑΟεۈ຿
15. SREʹมߋͨ͠ཧ༝ • ʮΠϯϑϥʯ͔ΒΠϝʔδ͢Δۀ຿ͷ࿮Λ௒͑Δ • ϝϧΧϦΛ5೥ɾ10೥ͱ௕͘࢖͍ͬͯͨͩͨ͘Ίͷʮ৴པੑʯ • εϚϗͰ͸24࣌ؒ޷͖ͳ࣌ʹɺ޷͖ͳ৔ॴ͔ΒΞΫηε͕དྷ Δɻ͍ͭͰ΋շదʹΞΫηε͕Ͱ͖ΔʮՄ༻ੑʯͱ
 ʮύϑΥʔϚϯεʯ • ιϑτ΢ΣΞΤϯδχΞϦϯάͰ͜ΕΒΛղܾ͍ͯ͘͠νʔϜ • (ւ֎/USͰ΋௨͡Δ໊শ)
16. Mercari SREͷۀ຿ • APIαʔόɺϛυϧ΢ΣΞͷՄ༻ੑɺύϑΥʔϚϯεͷ
 ҡ࣋ɾ޲্ • OnCall (ো֐ରԠͷ౰൪) • ϩάऩूɾ෼ੳج൫ͷߏஙɺӡ༻ • αʔόϓϩϏδϣχϯάɾσϓϩΠͷ੔උ • ηΩϡϦςΟͷ୲อ
17. Mercari SREࣄྫ • Gaurun - push notification server • ngx_luaʹΑΔBalancer֦ு • PHP daemon class preload • URL Shorten Service • ChatOpsˍZero DownTime Deploy • etc…
18. ChatOps Deploy
19. ChatOps Deploy • ϝϧΧϦ • SCMͱͯ͠GitHubΛ࠾༻ • IssueTrackerͰͷ࢓༷ݕ౼ • GitHub PullRequestͰͷϐΞϨϏϡʔ
20. Google Calendar + ChatOps
21. Inside ChatOps @jp-go-bold: yes check PullRequest check Issue Tracker exec Ansible merge PR git clone git clone composer install deploy application preprocessing ϐΞϨϏϡʔࡁ Ϛωʔδϟঝೝࡁ
22. ·ͱΊ
23. ·ͱΊ • ϝϧΧϦ͸C2CϚʔέοτϓϨΠεɺ৴པੑॏཁ • SRE͸৴པੑΛγεςϜ໘͔Βࢧ͑Δ • ۀ຿͸Πϯϑϥʹཹ·Βͳ͍ɻ৴པੑΛ௥ٻ