Upgrade to Pro — share decks privately, control downloads, hide ads and more …

検索基盤移行時の思考

po3rin
August 22, 2022

 検索基盤移行時の思考

po3rin

August 22, 2022
Tweet

More Decks by po3rin

Other Decks in Programming

Transcript

  1. ·ͨɺҩྍಛԽͷݕࡧΤϯδϯͷ࿦จ<>ʹݴٴ͕͋ΔΑ͏ʹҩྍݕࡧʹ͸ԼهͷࣾձతϝϦοτ͕ ͋Δɻ  ঱ঢ়ͷࣗݾ਍அΛ༰қʹ͢Δ  ҩࢣͱͷ໘ձલʹ঱ঢ়ʹؔ͢Δ஌ࣝΛಘΔ  ҩࢣͱͷ໘ձͷޙɺ׬શʹ͸ཧղ͍ͯ͠ͳ͍৘ใΛফԽ͢Δͷʹ໾ཱͯΔ͜ͱ͕Ͱ͖Δɻ ҩྍݕࡧͷॏཁੑ <>-VP(

    5BOH$ :BOH) 8FJ9.FETFBSDIBTQFDJBMJ[FETFBSDIFOHJOFGPSNFEJDBMJOGPSNBUJPOSFUSJFWBM*O1SPDFFEJOHTPGUIFUI "$.$POGFSFODFPO*OGPSNBUJPOBOE,OPXMFEHF.BOBHFNFOU r 4VTBOOBIΒ<>ʹΑΔͱˋҎ্ͷਓ͕ɺΦϯϥΠϯͰҩྍ৘ใΛݕࡧͨ͜͠ͱ͕͋Δɺࣗݾ਍அ Λ͢Δਓͷߦಈͷ͸ݕࡧΤϯδϯͰ࢝·Δ <>4VTBOOBI' .BFWF%)FBMUI0OMJOFIUUQTXXXQFXSFTFBSDIPSHJOUFSOFUIFBMUIPOMJOF
  2. ΤϜεϦʔͷݕࡧج൫Ҡߦ #FGPSF ҩࢣ޲͚ݕࡧ ױऀ޲͚ݕࡧ ΦϯϓϨ Ϋϥ΢υ w "*ɾػցֶशνʔϜͰӡ༻؅ཧ w ਫ਼౓վળͷͨΊʹఆظతʹϝϯςφϯε

    w ܗଶૉղੳʹ4VEBDIJΛ࠾༻ w ผνʔϜͰӡ༻؅ཧ w ݕࡧਫ਼౓͕՝୊ͩͬͨ w ܗଶૉղੳʹ͸σϑΥϧτͷLVSPNPKJΛ࠾༻
  3. Ҡߦ͢Δݕࡧج൫Ͱ࢖ΘΕ͍ͯΔݕࡧΫΤϦΛҠߦલʹ֬ೝ͓ͯ͘͠ͷ͕؊ w ΫΤϦ૯਺ɺΫΤϦ૯ϢχʔΫ਺ w ΫΤϦʹग़ݱ͢Δ඼ࢺͷ෼෍ w λʔϜ͝ͱͷग़ݱ཰ w λʔϜ਺ͷग़ݱ཰ w

    λʔϜ௕ͷ౷ܭྔ ௕͍ͱෳ߹ޠ໰୊͕ൃੜ͢ΔڪΕ͕͋Δ  w θϩώοτΫΤϦ૯਺ ্ه͸΋ͪΖΜɺ໨ࢹͰ֬ೝ͍ͯ͘͜͠ͱͰݕࡧج൫͝ͱͷϢʔβʔͷݕࡧಈ޲ͷಛ௃Λ௫Ήɻ͔͜͜ ΒΧελϜࣙॻ΍γϊχϜࣙॻߏஙɺΫΤϦߏங࣌ʹඞཁͳ৘ใ͕ݟ͑ͯ͘Δ ͜͜·Ͱͷ໰୊Λ౿Έൈ͔ͳ͍ͨΊͷࢥߟ
  4. Ҡߦཁ݅ݕࡧ඼࣭ SFDBMM ͕མ͍ͪͯͳ͍͜ͱ  ɾαϯϓϦϯάͨ͠ΫΤϦ݅ͷώοτ਺Ͱલޙൺֱʹେ͖͕ࠩ͘ͳ͍͜ͱΛ֬ೝ ɾ݅ͷதͷΫΤϦ݅Λ඼ࢺͳͲ͕ຊ൪ΫΤϦͱʹͳΔΑ͏ʹαϯϓϦϯά͠ɺ໨ࢹͰ࣮ࡍ ͷ඼࣭Λൺֱ͠ҧ࿨ײ͕ͳ͍͜ͱΛ֬ೝ ࠓ·Ͱ໰୊ʹͳ͍ͬͯͨΫΤϦ͸ඞؚͣΉ ݕࡧج൫Ҡߦཁ݅ఆٛͷྫ ཁ݅

    આ໌ ख๏ ΫΤϦ50000݅Ͱɺώοτ਺͕20%Ҏ ্ݮΔ΋ͷɺ΋͘͠͸৽ͨʹθϩ݅ʹ ͳͬͨ΋ͷͷ਺͕1%ҎԼ ͓͔͠ͳݕࡧͰͨ·ͨ·ώοτ͕ଟ͘ग़͍͚ͯͨ ͲɺҠߦͰݮͬͯ͠·ͬͨ΋ͷ͸Χ΢ϯτ͠ͳ͍ ώοτ਺ܭଌπʔϧʹΑΔूܭͱ໨ࢹ ΫΤϦ50000݅Ͱɺݕࡧ݁Ռ͕5݅Ҏ ԼͩͬͨΫΤϦͷ਺͕ݮ͍ͬͯΔɻ ͓͔͠ͳݕࡧͰͨ·ͨ·ώοτ͕ଟ͘ग़͍ͯΔͷ ͷ͸Χ΢ϯτ͠ͳ͍ ώοτ਺ܭଌπʔϧʹΑΔूܭͱ໨ࢹ ΫΤϦ5000݅Ͱɺ͓͔͠ͳཧ༝Ͱώο τ਺͕ݮͬͨ΋ͷ͕ͳ͍ɻ ໨ࢹ
  5. ΤϜεϦʔͰ৽ͨʹݕࡧվળ࣌ͷ౷ܭྔ֬ೝ໨ࢹ֬ೝπʔϧΛ։ൃ͠ɺલεϥΠυͷཁ݅Λຬͨ͞ͳ ͍΋ͷΛ͙͢ʹ֬ೝͰ͖ΔΑ͏ʹͨ͠ɻ1.͕֬ೝͰ͖ΔΑʹ$47ग़ྗΛߦͬͨɻώοτ਺͸΋ͪΖ Μɺ࣮ࡍͷϨεϙϯεͷ໨ࢹνΣοΫͷͨΊͷUPQLΛग़ྗɻ·ͨɺௐࠪͷͨΊʹΫΤϦͷܗଶૉղੳ݁ Ռ΋෇༩͓ͯ͘͠ͱݪҼΛಛఆ͠΍͍͢ɻ ྫ ݕࡧج൫Ҡߦ࣌ͷ໨ࢹ֬ೝπʔϧͷಋೖ Query hits_before hits_after topk_before

    topk_after tokens_before tokens_after warn ඓਫͰͳ͍෭ඓߢԌ͋Γ·͔͢ 4888 4885 ݕࡧ݁JSON(লུ) ݕࡧ݁JSON(লུ) ඓਫ / ෭ / ඓߢ / Ԍ ඓਫ / ෭ඓߢԌ 0 ศൿ 2103 3242 ݕࡧ݁JSON(লུ) ݕࡧ݁JSON(লུ) ศൿ ศൿ / ศൿ঱ 0 ௿݂ѹ঱ 6000 123 ݕࡧ݁JSON(লུ) ݕࡧ݁JSON(লུ) ௿݂ѹ঱ / ௿݂ѹ ௿݂ѹ঱ 1
  6. FTLFFQFSͰ͸Լهͷεςʔδʹ෼ׂ͞ΕΔ ઃఆϑΝΠϧͷόϦσʔγϣϯ ඥͮ͘NBQQJOH fi MF͸ଘࡏ͢Δ͔ ඞਢϑΟʔϧυ͸ຬͨ͞Ε͍ͯΔ͔ ࣄલνΣοΫ JOEFY͕࣮ࡍʹ࡞ΕΔ͔ Ұ࣌తͳJOEFYΛ࣮ࡍʹ࡞੒ 

    BMJBTͱʹඥͮ͘JOEFY͸ଘࡏ͢Δͷ͔ ࣮ߦεςʔδ ࣮ࡍʹఆٛϑΝΠϧͷঢ়ଶΛಉظͤ͞Δ ࣄޙνΣοΫ ఆٛϑΝΠϧʹ͋ΔJOEFY΍BMJBT͕࡞ΒΕ͔ͨ