'D'evelopment and 'D'eployment with 'D'ocker at 'D'wango

Development and Deployment with Docker at Dwango @ixixi

%8"/(0$P -UE :VSJ0EBHJSJ !JYJYJ 3FDPNNFOEBUJPOTZTUFNBSDIJUFDU

Development with Docker

/JDPOJDP "QQMJDBUJPO4FSWFS )551SFRVFTU 8FC 4FSWFS $BTTBOESB 3FDPNNFOE  "1*4FSWFS Recommendation API
3FDPNNFOE 4FSWFS 0UIFS4FSWJDFT MPHT $BDIF4FSWFS MPHT w%PDLFSҎલ͔ΒՔಇ͍ͯ͠ΔϨίϝϯυ"1* wχίχίಈըͷʮؔ࿈ಈըʯͳͲΛฦ͢ɻ

Ϩίϝϯυͷ֬ೝͱςετ wϨίϝϯυͷγεςϜʹखΛՃ͑ͨ৔߹ʹɺϨίϝϯυ݁Ռ͕Ͳ͏  มԽ͢Δ͔ͷ֬ೝ͸ɺίʔυϨϏϡʔɾ6OJU5FTUͰ͸อূͮ͠Β͍ɻ ! wQVMMSFRΛNBTUFSʹऔΓࠐΉຊ൪σϓϩΠͳͷͰɺ֬ೝແ͠Ͱͷ౤ ೖ͸ͨ͘͠ͳ͍ɻ ! wͲ͏ʹ͔ͯ͠ɺखܰʹʮ࣮ࡍͷ؀ڥΛ֬ೝʯग़དྷΔΑ͏ʹ͍ͨ͠ɻ

΍Γ͍ͨ͜ͱ wQVMMSFRग़ͨ͠ΒɺϨίϝϯυʹؔΘΔ%PDLFSͷ  ίϯςφҰἧ͍ lܥz ͕উखʹཱͭɻ wࣗ෼Ͱ؀ڥΛ࡞ͬͯࢼ͞ͳͯ͘΋ɺ  ʮ͜͜Ͱطʹಈ͍͍ͯΔΑʯͱ͍͏ঢ়ଶ͕࠷ߴɻ ! wϨίϝϯυ"1*Ͱɺ࣮ࡍʹԿ͕Ϩίϝϯυ͞ΕΔ͔  දࣔग़དྷΔ3FDPNNFOE7JFXFS
಺෦πʔϧ Ͱݟ͍ͨɻ طʹಈ͍͍ͯΔ

/JDPOJDP "QQMJDBUJPO4FSWFS )551SFRVFTU 8FC 4FSWFS $BTTBOESB 3FDPNNFOE  "1*4FSWFS Containerize! 3FDPNNFOE
4FSWFS 0UIFS4FSWJDFT MPHT $BDIF4FSWFS MPHT 3FDPNNFOE 7JFXFS )551SFRVFTU

pull request builderͰҰ্ཱࣜͪ͛&௨஌ wKFOLJOTͷQVMMSFRVFTUCVJMEFSΛར༻ɻ wKPCDIBJOͰςετ͔Βίϯςφ࡞੒ɺίϝϯτॻ͖ࠐΈ·Ͱ࣮ߦɻ QVMMSFRVFTUCVJMEFS QVMMSFRVFTU ௨ৗͷςετ QVMMSFRVFTUΛ൓өͨ͠ EPDLFSίϯςφ࡞੒
EPDLFSίϯςφͰ ܥΛҰࣜىಈ ઀ଓઌϙʔτ৘ใΛ QVMMSFRίϝϯτʹॻࠐ ίϝϯτʹॻ͖ࠐΈ

ίϯςφੜ੒ wQVMMSFRVFTUʹରԠͨ͠ίϯςφ໊ͷQSFpYΛ࡞Δɻ  wྫ QVMMSFR൪߸͕Ͱ࣌ࠁ͕ͳΒ  ɹɹQVMMSFR@SFDPNNFOE@BQJ@@U  wಉҰQVMMSFR಺Ͱ΋ɺߋ৽͞ΕΔ౓ʹίϯςφੜ੒͍ͨ͠ͷͰɺ೔͕࣌ඞཁɻ  wQVMMSFRऴྃ࣌ ޙड़ ʹ͜ͷQSFpYΛར༻ɻ

ίϯςφͰʮܥʯΛཱͯΔ wHJUQVMM͔ͯ͠ΒpHCVJME͢ΔSBLFUBTLΛ࣮ߦ wQVMMSFRʹରԠͨ͠CSBODIΛ࢖ͬͯCVJME͕૸Δɻ wΞϓϦߋ৽෦෼Ҏલ͸Ωϟογϡ͕ޮ͘ͷͰCVJME͸ߴ଎ ! wੜ੒ͨ͠QSFpYΛ෇Ճͨ͠ίϯςφ໊ͰpHVQ͢ΔSBLFUBTL࣮ߦ ! wDBTTBOESBSFEJTͷαʔό͸ɺμϛʔσʔλ͕  ೖͬͨίϯςφ্ཱ͕͕ͪΔɻ !
wQVMMSFRͷίϝϯτͰɺىಈͤ͞Δ֤ίϯςφͷ  CSBODIΛ໌ࣔࢦఆ͢Δ͜ͱ΋Մೳɻ  ผͷSFQPTJUPSZͷQVMMSFR΋Ұॹʹద༻͢Δඞཁ͕͋Δ৔߹౳ $BTTBOESB 3FDPNNFE  "1*4FSWFS 3FDPNNFOE 4FSWFS $BDIF 4FSWFS 3FDPNNFOE 7JFXFS MJOL MJOL MJOL MJOL

PullReqίϝϯτ΁ͷॻ͖ࠐΈ w֤ίϯςφׂ͕Γ౰ͯΒΕͨϙʔτΛऔಘɻ ! w઀ଓ৘ใΛ()FͷQVMMSFRίϝϯτʹॻ͘ɻ

Garbage Collection (daily) wQVMMSFRVFTU͕DMPTF͞ΕͨΒίϯςφΛࢭΊΔ࡟আ͢Δɻ  wKFOLJOTͰQVMMSFRVFTUͷDMPTFIPPL͢Δ͜ͱ͕೉͔ͬͨͨ͠Ίɺ೔࣍Ͱ($ɻ DSPO   wEPDLFSͷίϯςφ໊ͷQSFpYΛQBSTFͯ͠ɺSFQPTJUPSZͱQVMMSFRVFTU൪߸Λऔಘɻ  w౰֘QVMMSFRVFTU͕DMPTF͞Ε͍ͯΔ͔֬ೝ͠ɺด͍ͯͨ͡Βίϯςφఀࢭ࡟আɻ !
wίϯςφ໊ͷQSFpYͰ͸ͳ͘FUDE΍SFEJTʹରԠΛอ࣋ͯ͠΋ྑ͔͕ͬͨɺߏ੒ͷγϯϓ ϧ͞Λॏࢹɻ

developmentฤ Ή͢ͼ wطଘͷϓϩμΫτΛ%PDLFSԽ͢Δͷ͸ਏ͍৔߹͕͋ͬͨɻ w؀ڥߏஙΛDIFGͰߦ͏લఏͷϓϩμΫτΛޙ͔Β%PDLFSԽ͢Δͷ͸େมʹײͨ͡ɻ ! wطଘ෦෼Λ%PDLFSԽ͢ΔΑΓɺ৽نͰ࡞Δ΋ͷΛ%PDLFSԽલఏͰߏங͢Δํ͕ྑ͍ɻ w1VMM3FRVFTUͰʮͦͷ৔Ͱ࣮ࡍʹ৮ΕΔ؀ڥ͕طʹಈ͍͍ͯΔʯͷ͸࠷ߴɻ

Deployment with Docker

Personalize API /JDPOJDP "QQMJDBUJPO4FSWFS )551SFRVFTU 8FC 4FSWFS $BDIF %BUBCBTF 3FWFSTF
1SPYZ 1FSTPOBMJ[F"1* • ύʔιφϥΠζʹར༻͢ΔAPI (golang࣮૷) • DockerલఏͰ৽نʹߏஙɻ

Containerize! • ύʔιφϥΠζAPI • ϩάసૹ&ू໿(fluentd) • CEPΤϯδϯ(Norikra) • ϩάอଘ (ES&InfluxDB)
• ՄࢹԽ (Kibana&Grafana) • ϦόʔεϓϩΩγ 1FSTPOBMJ[F"1* /JDPOJDP "QQMJDBUJPO4FSWFS )551SFRVFTU $POUBJOFSJ[F 8FC 4FSWFS $BDIF %BUBCBTF MPHTUPSF .FUSJDT7JTVBMJ[FS $&1 -PH$PMMFDUPS 3FWFSTF 1SPYZ

Dataflow DP 4FSWFS )551SFRVFTU $BDIF %BUBCBTF MPHTUPSF .FUSJDT7JTVBMJ[FS $&1 3FWFSTF
1SPYZ -PBE #BMBODFS -PH"HHSFHBUPS MPHpMF %PDLFS)PTU %PDLFS)PTU %PDLFS)PTU %PDLFS)PTU "1* -PH $PMMFDUPS w ֤IPTU಺ͷϦόʔεϓϩΩγ OHJOY ܦ༝Ͱ"1*ʹϦΫΤετɻ w ϩάϑΝΠϧ͸ϗετଆϘϦϡʔϜΛϚ΢ϯτͯ͠ॻ͖ग़͠ɻ w qVFOUEίϯςφଆͰ΋ϩάσΟϨΫτϦΛϚ΢ϯτͯ͠ɺಡΈࠐΈɻ w ϩάू໿ίϯςφͰϩάΛूΊͯɺ$&1 /PSJLSB Ͱूܭͭͭ͠ʹอଘˠՄࢹԽ

Docker Build Architecture MPDBM ()F +FOLJOT 3FHJTUSZ $PNNPO QVTITPVSDFT )PPL
w %PDLFS*NBHFϏϧυ w ίϯςφͰܥΛ্ཱͪ͛ͯ݁߹ςετ w υΩϡϝϯτͷੜ੒ w %PDLFS3FHJTUSZʹΠϝʔδొ࿥ "1* qVFOUE /HJOY SFHJTUFS"1*DPOUBJOFS*NBHF VCVOUV CVJMEͷى఺ͱͳΔJNBHF͸ɺ1SJWBUF SFHJTUSZʹஔ͍͓͖ͯɺ%PDLFSIVC SFHJTUSZͷ౎౓ར༻͸͠ͳ͍ɻ

Deploy Manager WebUI • WebUIσϓϩΠπʔϧ • DockerԽҎલ͔Β࢖͍ͬͯͨ಺੡πʔϧ • Capistrano task
࣮ߦ • Ϣʔβೝূ • σϓϩΠϩάͷϦΞϧλΠϜදࣔ&อଘ  • DockerσϓϩΠ΋capistrano taskԽͯ͠ར༻  σϓϩΠઌαʔόΛબ୒ͯ͠ɺϙνοͱ  1click͚ͩͰແఀࢭσϓϩΠɻ • σϓϩΠͷrollbackλεΫ΋༻ҙɻ

Docker Deployment Architecture %FQMPZ.BOBHFS 8FC6* 3FHJTUSZ ڞ௨ 3FHJTUSZ  QSPEVDUJPOઃఆ "1*
qVFOUE /HJOY "1*TFSWFSDPOpH 0QFSBUF QVMM qVFOUE 3FWFSTF1SPYZ "1* QVMM "1*TFSWFSDPOpH %PDLFS)PTU 0UIFS4FSWJDFT %FQMPZ $BQJTUSBOPUBTL )551SFRVFTU NPVOU • Capistrano taskͰɺίϯςφͷ  ΠϝʔδΛ֤ϗετ͕pullͯ͠σϓϩΠɻ ! • ࣮ߦόΠφϦͷίϯςφΠϝʔδ͸։ൃऀ શһ͕pullՄೳɻ ! • production؀ڥͰ࢖͏configίϯςφ͸ɺ registry͝ͱ෼཭(ೝূ͕ҟͳΔ)ɻ

Deployment ~ step.1 1.APIίϯςφΠϝʔδͱɺઃఆΠϝʔδ ΛɺDockerHostଆͰpull͢Δ %PDLFS)PTU 3FHJTUSZ ڞ௨ 3FHJTUSZ ઃఆ
"1* qVFOUE /HJOY "1*W EPDLFSJNBHFT "1*TFSWFSDPOpH "1*TFSWFSDPOpH "1*TFSWFS DPOpH NPVOU NPVOU 3FWFSTF1SPYZ "1*W qVFOUEMBUFTU

Deployment ~ step.1 1.APIίϯςφΠϝʔδͱɺઃఆΠϝʔδ ΛɺDockerHostଆͰpull͢Δ %PDLFS)PTU 3FHJTUSZ ڞ௨ 3FHJTUSZ ઃఆ
"1* qVFOUE /HJOY EPDLFSQVMMBQJMBUFTU EPDLFSQVMMBQJ@DPOpHMBUFTU "1*W EPDLFSJNBHFT "1*W MBUFTU "1*TFSWFSDPOpH "1*TFSWFSDPOpH "1*TFSWFS DPOpH NPVOU NPVOU 3FWFSTF1SPYZ "1*W qVFOUEMBUFTU

Deployment ~ step.2 1.৽͍͠imageͷίϯςφΛىಈ 2.API͕ਖ਼͘͠ϦΫΤετΛฦͤΔ·Ͱ଴ͭ  ɹ(࣮ࡍʹϨεϙϯεΛ֬ೝ) %PDLFS)PTU "1*W qVFOUEMBUFTU DPOpH
3FWFSTF1SPYZ

Deployment ~ step.2 1.৽͍͠imageͷίϯςφΛىಈ 2.API͕ਖ਼͘͠ϦΫΤετΛฦͤΔ·Ͱ଴ͭ  ɹ(࣮ࡍʹϨεϙϯεΛ֬ೝ) %PDLFS)PTU "1*W "1*W qVFOUEMBUFTU
qVFOUEMBUFTU DPOpH DPOpH 3FWFSTF1SPYZ

Deployment ~ step.2 1.৽͍͠imageͷίϯςφΛىಈ 2.API͕ਖ਼͘͠ϦΫΤετΛฦͤΔ·Ͱ଴ͭ  ɹ(࣮ࡍʹϨεϙϯεΛ֬ೝ) %PDLFS)PTU "1*W "1*W qVFOUEMBUFTU
qVFOUEMBUFTU DPOpH DPOpH IFBMUIDIFDL EVNNZSFRVFTU 3FWFSTF1SPYZ

Deployment ~ step.3 1.৽͍͠IPΛnginxͷconfʹॻ͖ࠐΉɻ 2.NginxΛgraceful restartͯ͠ɺແఀࢭͰίϯςφ ͷ޲͚ઌΛมߋ͢Δɻ  Reverse Proxy(nginx)ͷDockerίϯςφ಺Ͱ 1.$
kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`        ※࣮ࡍʹ͸ɺgracefull restart෦෼͸  γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS)PTU "1*W "1*W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF1SPYZ DPOpH DPOpH OHJOYDPOG OHJOY QPSU

kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`        ※࣮ࡍʹ͸ɺgracefull restart෦෼͸  γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS)PTU "1*W "1*W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF1SPYZ DPOpH DPOpH OHJOYDPOG MPDBUJPO\ QSPYZ@QBTTIUUQ ^ OHJOY QPSU

kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`        ※࣮ࡍʹ͸ɺgracefull restart෦෼͸  γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS)PTU "1*W "1*W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF1SPYZ DPOpH DPOpH OHJOYDPOG MPDBUJPO\ QSPYZ@QBTTIUUQ ^ MPDBUJPO\ QSPYZ@QBTTIUUQ ^ OHJOY QPSU

kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`        ※࣮ࡍʹ͸ɺgracefull restart෦෼͸  γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS)PTU "1*W "1*W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF1SPYZ DPOpH DPOpH OHJOYDPOG OHJOY OHJOY QPSU

Deployment ~ step.4 1.ݹ͍ίϯςφͷఀࢭ&࡟আ 2.ݹ͍ίϯςφΠϝʔδͷ࡟আ    ˞ߴ଎ʹrollbackग़དྷΔΑ͏ɺ࣮ࡍʹ͸  ɹ5ੈ୅લͷΠϝʔδΛ࡟আ %PDLFS)PTU "1*W
"1*W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF1SPYZ DPOpH DPOpH OHJOY "1*W EPDLFSJNBHFT "1*W MBUFTU "1*TFSWFSDPOpH

qVFOUEMBUFTU 3FWFSTF1SPYZ DPOpH OHJOY "1*W EPDLFSJNBHFT "1*W MBUFTU "1*TFSWFSDPOpH

qVFOUEMBUFTU 3FWFSTF1SPYZ DPOpH OHJOY EPDLFSJNBHFT "1*W MBUFTU "1*TFSWFSDPOpH

deploymentฤ Ή͢ͼ wμ΢ϯλΠϜແ͠ͷσϓϩΠ͸Մೳ͕ͩɺݱঢ়ɺࣗલͰগ͠࡞Γ͜Ήඞཁ͕͋Δɻ wࣗલͷίϯςφ੾ସલϔϧενΣοΫɻ ! wϗετଆͷ؀ڥ͕ԚΕͳ͍ͷ͸ૉ੖Β͍͠ɻ ! w௕ظӡ༻͢ΔલఏͳΒΰϛΛย෇͚ΔػߏΛ࣋ͱ͏ɻ  wLTΛ࢖͏ʹ͸ɺNBTUFSͷ৑௕Խ΍ߴՄ༻ੑͳFUDEΫϥελͷߏஙɾӡ༻౳ɺ  ن໛ͷׂʹ߹Θͳ͔ͬͨͷͰʮϦιʔεͷந৅Խʯ·Ͱ͸౿Έࠐ·ͳͳ͔ͬͨɻ 
wαʔϏεԣஅͰར༻Մೳͳେن໛σϓϩΠઌαʔόࢿݯͱͯ͠ந৅Խ͢Δ৔߹ʹ͸ɺ  LT౳͕ඞཁʹͳΔͱࢥ͏ɻن໛ͱ໨తʹԠͯ͡ΞʔΩςΫνϟઃܭ͠Α͏ɻ

'D'evelopment and 'D'eployment with 'D'ocker at...

'D'evelopment and 'D'eployment with 'D'ocker at 'D'wango

ixixi

More Decks by ixixi

Other Decks in Technology

Featured

Transcript