Share
Embed
MLOpsの「壁」を乗り越える、LINEヤフーの Data Quality as Code
データの品質はMLモデルの信頼性に関わる重要な指標ですが、サイエンティストと運用者の連携は容易ではなく、MLOpsをスケールアウトする上で運用面のボトルネックとなりやすいポイントです。このセッションでは、信頼性の高いMLモデル運用を効果的に実現する、LINEヤフーのData Quality as Codeの仕組みを紹介します。
LINEヤフーTech (LY Corporation Tech) PRO
April 23, 2024
Transcript
26 slides
-
-:$PSQPSBUJPO ӡ༻ͷ৴པੑ .- 0QT .-0QTશମͱͯ͠ ։ൃͷੜ࢈ੑΑΓӡ༻ͷ৴པੑ͕ εέʔϧͷϘτϧωοΫʹͳ͖ͬͯͨ ࣄނͷ૿Ճ 55%ɾ55.ͷظԽ
˞55%ɿ5JNFUP%FUFDUʢࣄނݕग़࣌ؒʣ ˞55.ɿ5JNFUP.JUJHBUFʢࣄނܰݮ࣌ؒʣ -
-:$PSQPSBUJPO .-0QTͷߏత՝ .- 0QT ӡ༻ͷ৴པੑΛԼ͍ͤͯ͞Δσʔλࣄނͷ55%ɾ55.͕ظԽ͢Δͷ ։ൃऀӡ༻ऀͷೳྗ͕ݪҼͰͳ͘ ։ൃऀͱӡ༻ऀͰσʔλ࣭͕͏·͘ڞ༗Ͱ͖͍ͯͳ͍͜ͱ͕ݪҼ σʔλ࣭͕Θ͔Βͳ͍ 55%ɾ55.͕
ѱԽ͍͢͠ σʔλ࣭ͷཁ݅ఆ͕ٛ ෆेͳ··ϦϦʔε ͞Ε͍͢ σʔλ࣭վળͷ ίϛϡχέʔγϣϯϩε͕ ى͖͍͢ อकతͳ࣭ཧʹ େ͖ͳΛׂ͚ͳ͍ -
-:$PSQPSBUJPO "$1%BUB2VBMJUZ "$1%BUB2VBMJUZ*40ʹجͮ͘σʔλ࣭ཧϓϩηεߏஙΛࢧԉ͢ΔαʔϏε ։ൃऀͱӡ༻ऀ͕ίϥϘϨʔγϣϯͯ͠αεςφϒϧͳσʔλ࣭ཧΛߏஙͰ͖Δ σʔλ࣭ܭը %BUB2VBMJUZ1MBOOJOH σʔλ੍࣭ޚ %BUB2VBMJUZ$POUSPM σʔλ࣭อূ
%BUB2VBMJUZ"TTVSBODF σʔλ࣭վળ %BUB2VBMJUZ*NQSPWFNFOU σʔλ࣭ཧϓϩηε Ϧιʔεنఆ σʔλؔ࿈αϙʔτ ˞"$1ɿ͚ࣾʹఏڙ͍ͯ͠Δ"*ಛԽͷ,VCFSOFUFTڥɻ -
-:$PSQPSBUJPO %2.- %BUB2VBMJUZ.PEFM-BOHVBHF σʔλ࣭Ϟσϧ Λ ίʔυͰఆٛ͢ΔͨΊͷݴޠΛ։ൃ ࣭ ཁ
ٻ ର σʔλ9 త Ϟσϧ.ͷֶश༻σʔλ ཁٻ ͋ΔϞσϧ.ͷֶशʹ͏σʔλ9ɺ༗ޮͳֶश݁ՌΛಘΔͨΊʹ ݅Ҏ্ͷσʔλ͕ͳ͚ΕͳΒͳ͍ ࣭ ཁ ݅ ଌఆؔ " ଌఆྔཁૉ" σʔλ9ͷ݅ ଌఆج४ ݅ΑΓଟ͘ͳ͚ΕͳΒͳ͍ $PVOU NVTUCF ͜ͷʮσʔλ࣭Ϟσϧʯͱ l*40*&$4PGUXBSFFOHJOFFSJOH 4PGUXBSFQSPEVDU2VBMJUZ3FRVJSFNFOUTBOE&WBMVBUJPO 42VB3& %BUBRVBMJUZNPEFMz Ͱఆٛ͞ΕΔͷΛࢦ͢ɻ -
-:$PSQPSBUJPO %2.-ʹΑΔཧϓϩηε .- 0QT ੍ޚ อূ ܭը վળ ίʔυʹΑΔཁ݅ఆٛ
ίʔυϨϏϡʔ ίʔυ͔ΒࣗಈධՁ ίʔυʹΑΔҰ؏ੑ -
-:$PSQPSBUJPO %2.-ͷޮՌ .- 0QT σʔλ࣭ͷఆ͔ٛΒ ࣗಈͰ੍ޚ 55%ɾ55.ͷվળ σʔλ࣭ͷཁ݅Λ ίʔυͰఆٛ͢Δ͚ͩͰ
Ռͷ࣭Λ্ ίʔυϕʔεͰ ࣭ͷཧͱվળΛ ܧଓతʹ࣮ࢪ ίʔυϕʔεͰ ࣭ཁ݅ͷվળ͚ͩʹूத ඞཁ࠷খݶͷͰӡ༻ʹߩݙ %BUB$FOUSJDͳϓϩηεʹΑͬͯ.-0QTΛՃ -
-:$PSQPSBUJPO ࣭ܭը ཁ݅ఆٛ ,VCFSOFUFTΧελϜϦιʔεͱͯ͠σʔλ࣭ϞσϧΛఆٛ σʔλ࣭ཁ݅Λ%2.-ʹΑΔίʔυͰఆٛ͠ɺ:".-$6&ͳͲͷܗࣜͰཧ BQJ7FSTJPOEBUBRVBMJUZZBIPPDPKQWBMQIB LJOE %BUB2VBMJUZ.PEFM NFUBEBUB
OBNF FYBNQMF TQFD EBUB4QFD USJOP UBCMF USBJO@EBUB@Y SFRVJSFNFOUT DPEF$PVOUNVTUCF ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO ࣭ܭը ཁ݅ཧ (JU)VCͰόʔδϣϯཧɾ,VCFSOFUFTʹಉظ σʔλ࣭Ϟσϧ%BUB2VBMJUZ.PEFM %2. BQJ7FSTJPOEBUBRVBMJUZZBIPPDPKQWBMQIB LJOE %BUB2VBMJUZ.PEFM
NFUBEBUB OBNF FYBNQMF TQFD EBUB4QFD USJOP UBCMF USBJO@EBUB@Y SFRVJSFNFOUT DPEF$PVOUNVTUCF %2. FYBNQMF (JU)VC BQQMZ ,VCFSOFUFT $*$%Ͱݕূɾಉظ ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO ࣭ܭը ੩తղੳ "ENJTTJPO8FCIPPLͱ੩తղੳ "ENJTTJPO8FCIPPLͰཁ݅Λ੩తղੳ ߏจղੳɾܕνΣοΫͳͲɺཁ݅ͷݕূ͕Մೳ BQJ7FSTJPOEBUBRVBMJUZZBIPPDPKQWBMQIB LJOE %BUB2VBMJUZ.PEFM
NFUBEBUB OBNF FYBNQMF TQFD EBUB4QFD USJOP UBCMF USBJO@EBUB@Y SFRVJSFNFOUT DPEF $PVOUXIFSFBHFJTlBCDz NVTUCF %2. FYBNQMF BHFͷܕ͕OVNCFSͱTUSJOHͰিಥ͍ͯ͠Δ "ENJTTJPO8FCIPPLʹΑͬͯ $SFBUF6QEBUF࣌ʹղੳ ͠Τϥʔ͕͋Εૢ࡞ΛϒϩοΫ͢Δ WBMJEBUF %2.-$6&ݴޠʹΠϯεύΠΞ͞ΕͨܕγεςϜΛ࣮͓ͯ͠Γɺ ܕਪΛߦ͍σʔλ࣭ཁ݅ͱͯ͠ໃ६͕͋Δ߹ʹΤϥʔΛใࠂ͢Δɻ ͜ͷྫͰɺϑΟʔϧυ͕͔ͭจࣈྻͱ͍͏ཁ݅͋Γಘͳ͍ͷͰɺ ܕγεςϜOVNCFSͱTUSJOHͷিಥΛใࠂ͢Δɻ ,VCFSOFUFT (JU)VC ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO ੍࣭ޚ ධՁ ,VCFSOFUFTϦιʔεͰσʔλ࣭ධՁΛ࣮ߦ σʔλ࣭ධՁ%BUB2VBMJUZ&WBMVBUJPO %2& BQJ7FSTJPOEBUBRVBMJUZZBIPPDPKQWBMQIB LJOE %BUB2VBMJUZ&WBMVBUJPO
TQFD NPEFM3FG OBNF FYBNQMF %2& FYBNQMFYYYY %2. FYBNQMF ,VCFSOFUFT %2&ϦιʔεΛ࡞ ධՁରͷϞσϧΛࢀর SFG DSFBUF ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO ੍࣭ޚ ࣗಈଌఆ ࣭ධՁʹΑΔཁ݅ͷࣗಈଌఆ BQJ7FSTJPOEBUBRVBMJUZZBIPPDPKQWBMQIB LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD NPEFM3FG
OBNF FYBNQMF 4&-&$5DPVOU '30.USBJO@EBUB@Y %2. FYBNQMF %2& FYBNQMFYYYY 8' FYBNQMFYYYYYYYY TUBUVT SFTVMUT NFBTVSF$PVOU DSJUFSJB WBMVF DPNQMJBOU GBMTF %2&3 FYBNQMFYYYY %BUBTPVSDF ,VCFSOFUFT ࣗಈͰϫʔΫϑϩʔΛ࡞ σʔλιʔεʹ߹ΘͤͯΫΤϦͳͲΛੜ ཁ݅ͷධՁʹඞཁͳଌఆྔΛશͯࣗಈͰଌఆ ධՁ݁Ռͷө DSFBUF NFBTVSF TBWF ࣮ࡍͬͱෳࡶɺେྔͷଌఆྔΛ ॲཧͰ͖ΔΑ͏ʹ࠷దԽ͞ΕΔɻ ෳͷσʔλιʔεΛαϙʔτ͠ ࣾͷଟ͘ͷσʔλʹରԠ͍ͯ͠Δɻ ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO ੍࣭ޚ ΞϥʔτɾϨϙʔτ ධՁ݁Ռ͔ΒΞϥʔτɾϨϙʔτͷ࡞ BQJ7FSTJPOEBUBRVBMJUZZBIPPDPKQWBMQIB LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD NPEFM3FG
OBNF FYBNQMF TUBUVT SFTVMUT NFBTVSF$PVOU DSJUFSJB WBMVF DPNQMJBOU GBMTF %2& FYBNQMFYYYY 4MBDL %BUBCBTF 8FC"QQ ࣭ʹҟৗ͕͋Γ·͢ʂ $PVOUNVTUCFHPU ,VCFSOFUFT 4MBDLʹΞϥʔτ %#ʹଌఆྔධՁΛه աڈͷଌఆྔͷਪҠͷՄࢹԽ ධՁͷੳϨϙʔτΛӾཡͰ͖Δ BMFSU QFSTJTU ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO ੍࣭ޚ ࢹɾ੍ޚ ,VCFSOFUFTΧελϜϦιʔεͰσʔλ࣭ϞσϧΛఆٛ BQJ7FSTJPOBSHPQSPKJPWBMQIB LJOE $SPO8PSLGMPX TQFD TDIFEVMFl!EBJMZz
XPSLGMPX4QFD UFNQMBUFT OBNFEBUBRVBMJUZFWBM SFTPVSDF BDUJPODSFBUF NBOJGFTU c BQJ7FSTJPOEBUBRVBMJUZ LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD NPEFM3FG OBNF FYBNQMF %2& FWBMFYBNQMFYYYY 8' FWBMFYBNQMFYYYY TUBUVT SFTVMUT NFBTVSF$PVOU DSJUFSJB WBMVF DPNQMJBOU GBMTF BQJ7FSTJPOEBUBRVBMJUZ LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD NPEFM3FG OBNF FYBNQMF ,VCFSOFUFT ϫʔΫϑϩʔ͔Β%2&Λ࡞ ධՁ݁Ռʹج͍ͮͯذɺ੍ޚ ྫɿϩʔϧόοΫɺதࢭͳͲ DSFBUF DPOUSPM ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO ੍࣭ޚ طଘγεςϜͱͷ౷߹ ඪ४తͳ,VCFSOFUFT"1*Λͬͨ౷߹ %2& FWBMFYBNQMF 5BTL FWBMFYBNQMFYYYY TUBUVT
SFTVMUT NFBTVSF$PVOU DSJUFSJB WBMVF DPNQMJBOU GBMTF BQJ7FSTJPOEBUBRVBMJUZ LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD NPEFM3FG OBNF FYBNQMF ,VCFSOFUFT "JSGMPX GSPN BJSGMPXJNQPSU %"( EBH%"( EBH@JElBJSGMPXFWBMz FWBMVBUJPO %BUB2VBMJUZ&WBMVBUJPO0QFSBUPS OBNFlFWBMFYBNQMFz UBTL@JElFWBMFYBNQMFz NPEFM@OBNFlFYBNQMFz EBHEBH FWBMVBUJPO ,VCFSOFUFT"1*Λ͑ ҙͷ֎෦γεςϜʹ؆୯ʹ౷߹Ͱ͖Δ DSFBUF DPOUSPM ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO ࣭վળ ϨϏϡʔ ,VCFSOFUFTΧελϜϦιʔεͰσʔλ࣭ϞσϧΛఆٛ BQJ7FSTJPOEBUBRVBMJUZZBIPPDPKQWBMQIB LJOE %BUB2VBMJUZ.PEFM NFUBEBUB
OBNF FYBNQMF TQFD EBUB4QFD USJOP UBCMF USBJO@EBUB@Y SFRVJSFNFOUT DPEF$PVOUNVTUCF (JU)VC BQJ7FSTJPOEBUBRVBMJUZZBIPPDPKQWBMQIB LJOE %BUB2VBMJUZ.PEFM NFUBEBUB OBNF FYBNQMF TQFD EBUB4QFD USJOP UBCMF USBJO@EBUB@Y SFRVJSFNFOUT DPEF$PVOUNVTUCF DPEF.JOPGBHFNVTUCF ίʔυϕʔεͰίϛοτɾϨϏϡʔ αΠΤϯςΟετͱӡ༻ऀ͕ ඇಉظͰίϛϡχέʔγϣϯ͍͢͠ $*$% DPNNJU SFWJFX $*$%ͰͷಉظʹΑΓ ཁ͔݅Β੍ޚ·Ͱ࣭ͷҰ؏ੑΛอূ ϦιʔεͷϑΟʔϧυεϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ -
-:$PSQPSBUJPO %BUB2VBMJUZBT$PEF %2.- • ,VCFSOFUFT$VTUPN3FTPVSDF • େྔͷϦιʔεΛॲཧͰ͖ΔΑ͏ʹίϯτϩʔϥΛ࠷దԽɺ ઍΛΏ͏ʹ͑Δσʔλ࣭ϞσϧͷධՁΛ҆ఆͯ͠ॲཧ •
%2.-ʢ%BUB2VBMJUZ.PEFM-BOHVBHFʣ • ܕγεςϜΛඋɺఆٛஈ֊Ͱ੩తղੳʹΑΔݕূ͕Մೳ • ༷ʑͳσʔλιʔεΛαϙʔτ͢ΔίϯύΠϥΛఏڙ ͋ΒΏΔσʔλʹରͯ͠ଌఆͱධՁ͕Մೳ -
-:$PSQPSBUJPO %BUB2VBMJUZBTB4FSWJDF "$1%BUB2VBMJUZ • 6OJWFSTBM • υϝΠϯಛԽಛఆͷڥʹґଘ͠ͳ͍ • อकੑΧελϜϦιʔεͱ%2.-ʹΑΔએݴతͳఆٛ
• ൚༻ੑ,VCFSOFUFT"1*ʹ४ڌ͠ҙͷγεςϜʹ౷߹Մೳ • Ҡ২ੑҟͳΔڥͰಉ࣭͡ཁ݅ΛཁٻͰ͖Δ • ࣮ෆཁσʔλ࣭ཧͷ࣮ߦͷͨΊʹཁ݅Ҏ֎ͷ࣮͕͍Βͳ͍ • 4FSWFSMFTT • ଈ࣌ಋೖσʔλ࣭ཧΛਝʹಋೖ • ӡ༻ෆཁՃͷӡ༻࡞ۀͳ͠Ͱ࣭ཧʹूத • εέʔϥϏϦςΟͱߴՄ༻ੑ,VCFSOFUFTʹΑΔେنల։͕Մೳ