とある業務オペレーション自動化の話

StackOverflow DevDays in Japan 2015.10.09

Transcript

1. ͱ͋Δۀ຿ΦϖϨʔγϣϯ ࣗಈԽͷ࿩ StackOverflow DevDays 2015.10.09 @onigra_
2. onigra Yudai Suzuki / ླ໦༤େ onigra_ tenso, inc. / tensoגࣜձࣾ
3. • Web Application Developer • DevOps • Rubyist • My favorite episode of Joel on Software: My First BillG Review
4. tensoגࣜձࣾʹ͍ͭͯ
5. Cross Border E-Commerce
 ӽڥECαʔϏε
6. We buy at Japanese EC on your behalf and ship the items to your address overseas.
7. Monolithic PHP Application
8. Monolithic PHP Application Micro Ruby Application Micro Ruby Application Micro Ruby Application Micro Ruby Application
9. ։ൃͷܦҢ
10. ։ൃͷܦҢ • αʔϏε։࢝౰ॳʹ࡞ΒΕͨɺ୅ཧߪೖͰ࢖ ༻͞ΕΔγεςϜ͕͋ͬͨ • αʔϏε͕֦େ͠ɺࣗ෼͕ೖࣾͯ͠࡞Γ௚͢ ͜ͱʹͳͬͨ
11. Architecture
12. Application • Ruby (Ruby on Rails) • Capybara • PhantomJS (Poltergeist) • Sidekiq
13. Infrastructure • AWS • Ansible • Tsudura (Original Packer Clone) • Capistrano • Blue-Green Deployment
15. Blue-Green Deployment • Auto Scaling Groupʹର͢ΔELBͷ෇͚֎͕͠Ͱ͖ΔΑ͏ʹͳͬͨͷͰಋ ೖ • ௨ৗͷDeploy͸Capistranoɺϛυϧ΢ΣΞ౳ͷΞοϓσʔτΛߦͬͨࡍ ʹ͸BG Deploy • PackerͰAnsible͕࢖͍ͮΒ͔ͬͨΓɺLaunch Config࡞੒΍Auto Scaling Groupͷ੾ସΛࣗಈͰ΍Γ͔ͨͬͨͷͰࣗ࡞ͷPacker clone cliΛ࡞੒ • http://github.com/onigra/tsudura • https://speakerdeck.com/onigra/about-tsudura
16. Job͕΍͍ͬͯΔ͜ͱ • νΣοΫΞ΢τʹඞཁͳ৘ใΛࣗಈೖྗ • ෳ਺ͷ߲໨ͷத͔Βબ୒ͷ൑அ͕ඞཁͳ΋ͷ ͸ɺͪ͜ΒͰܾΊͨ༏ઌॱҐΛ΋ͱʹબ୒͠ ͨΓ͢Δ • ࣗࣾ؅ཧը໘ʹऔಘͨ͠৘ใΛࣗಈೖྗ
17. SidekiqͰJobΛ
 ฒྻ࣮ߦ͢Δࡍʹ • δϣϒ͝ͱʹηογϣϯ͕෼͔ΕΔΑ͏ʹ͢Δ • ෼͚ͳ͍ͱಉ࣮࣌ߦ࣌ʹΤϥʔ • CookieϑΝΠϧΛRunnerͷΠϯελϯε͝ͱʹ TempfileͰ࡞੒ • RunnerͷΠϯελϯε͝ͱʹCapybara::SessionΛ ࡞੒
18. class WorkerSession attr_reader :session def initialize tempfile = Tempfile.new(prefix, path) Capybara.register_driver :poltergeist do |app| Capybara::Poltergeist::Driver.new( app, phantomjs_options: [ "--cookies-file=#{tempfile.path}", ] ) end @session = Capybara::Session.new(:poltergeist) end end
19. class ExampleRunner def initialize(param) @param = param @capybara = WorkerSession.new end def run login access_to_checkout select_shipping_method select_payment_method submit ensure logout delete_session end end
20. class ExampleWorker include Sidekiq::Worker sidekiq_options queue: :example def perform(param) ExampleRunner.new(param).run end end
21. ։ൃ։͔࢝ΒϦϦʔε·Ͱ
22. ։ൃ։͔࢝ΒϦϦʔε·Ͱ • ։ൃ։͔࢝Β2ि͙ؒΒ͍ͰϦϦʔε • جຊ1ਓͰ։ൃɺΠϯϑϥɺӡ༻ • ϑΝʔετϦϦʔε͸Ұ෦ػೳͷΈ • ϑΝʔετϦϦʔεޙɺগͣͭ͠ଞͷਓʹख఻ͬͯ΋ΒͬͨΓ • ܧଓతʹϦϦʔεΛߦ͍ɺ࠷ऴతʹۀ຿ଆ͕΍Γ͍ͨ͜ͱΛશ෦࣮ݱɻͦ ͷܭըΛͪΌΜͱۀ຿୲౰ऀͱѲͬͨ • CIͱσϓϩΠͷ࢓૊ΈΛ͔ͬ͠Γ੔උͯͨ͠ͷͰɺҰਓͰ΋͏·͍ͬͨ͘
23. ӡ༻͕͸͡·ͬͯ
24. PhantomJS͕Ϋϥογϡ͢Δ
26. ϦϦʔε౰ॳ • ࣮ߦ݅਺: 600-900/Day • ࣦഊ݅਺: 10-100/Day • PhantomJSͷΫϥογϡ͸Auto HealingͰޡ ຐԽͭͭ͠ӡ༻
27. ৽ػೳϦϦʔε
28. ৽ػೳϦϦʔεલ • ࣮ߦ݅਺: 1200-2000/day • ࣦഊ݅਺: 100-200/day
29. ϦϦʔεޙ • ࣮ߦ݅਺: 5000-6000/day • ࣦഊ݅਺: 600-1500/day • Ұ൪ଟ͍࣌Ͱ12000/5000
30. ϝϞϦϦʔΫʹΑΔCrashଟൃ • δϣϒ͕ϑϦʔζͨ͠ΠϯελϯεΛ௵ͯ͠ ͸ཱͯΔӡ༻ͷ೔ʑ • ΠϯελϯεαΠζ্͛ͨΓɺΠϯελϯε ਺૿΍ͯ͠΋େ͖ͳޮՌ͸ແ͠
31. (❨›°□°)❩›︵ᵲᴸᵲ
32. ֘౰ػೳͷ࠶ઃܭ • ແବʹ࣮ߦͯ͠͠·ͬͯΔδϣϒ͕ଟ͍ • ঢ়ଶΛݟͯɺແବͳδϣϒ͸࣮ߦ͞Εͳ͍Α ͏ʹͨ͠ • ݟͯݟ͵;Γͯͨ͠ϝϞϦϦʔΫʹຊ֨తʹ ޲͖߹͏
33. ϝϞϦϦʔΫରࡦ
34. https://github.com/teampoltergeist/ poltergeist#memory-leak PoltergeistͷREADME
35. session.driver.quit • ηογϣϯ࡞੒࣌ʹىಈͨ͠PhantomJSͷϓ ϩηεΛKILL͢Δϝιου͕༻ҙ͞Εͯͨ • ݁ہ΍ͬͨͷ͸͜ΕΛRunner#runϝιουͷ ensureʹೖΕ͚ͨͩ
39. class ExampleRunner def initialize(param) @param = param @capybara = WorkerSession.new end def run login access_to_checkout select_shipping_method select_payment_method submit ensure logout delete_session end end ͜͜΋ବ໨
40. νϡʔχϯάͷ݁Ռ • ॲཧ݅਺: 5000-7000/day • ࣦഊ݅਺: 20-50/day • Ϋϥογϡൃੜ͠ͳ͘ͳͬͨ • ϏϏͬͯಉ࣮࣌ߦ਺Λগͳ͍͕ͯͨ͘͠ɺ૿ ΍ͯ͠΋໰୊ͳ͘ͳͬͨ
42. ݱࡏ๊͍͑ͯΔ՝୊
43. CI͕͏·͍͔͘ͳ͘ͳͬͨ • CI͸CircleCI • ๺ถ͔ΒͷΞΫηεʹͳΔͷͰɺςετ಺ͰԿ౓΋Login܁ Γฦ͢ͱɺαʔϏεʹΑͬͯ͸Captureڬ·Εͯ٧Ή • ։ൃ౰ॳ͸େৎ෉ͩͬͨʢςετέʔεগͳ͔͔ͬͨΒʁʣ • ࠃ಺ͷαʔόʹDrone.ioཱͯΔͱ͔ɺࠃ಺ͷCIαʔϏε (SideCI)࢖͏ͱ͔ݕ౼த
44. ౰વ͕ͩɺ֎෦αʔϏεͷ
 ϚʔΫΞοϓ͕มΘͬͨΒେม • ͦΕΛݕ஌͢ΔͷΛؚΊͯɺ1೔ʹ਺ճςετ ࣮ߦͯͨ͠Μ͚ͩͲ… • ͱΓ͋͑ͣࣗ෼ͷϚγϯͰ1೔ʹԿճ͔࣮ߦ͠ ͯΔ • ࠓͷॴൃੜ͍ͯ͠ͳ͍
45. ࠷ޙʹ
46. ࠷ޙʹ • ͜Μͳײ͡Ͱ೔ʑ੒௕தͷαʔϏεΛεέʔ ϧͤ͞ΔͨΊʹؤு͍ͬͯ·͢ • ͦΜͳঢ়گͳͷͰɺฐࣾΤϯδχΞืूதͰ ͢ • ಛʹΠϯϑϥΤϯδχΞืूதͰ͢ʂ
47. Do you have any question?