日経のデータベース事業とElasticsearch

こちらのイベントでの登壇資料です。
Elasticsearch Community in Tokyo 2025
https://www.meetup.com/tokyo-elastic-fantastics/events/305751163/

Transcript

1. ೔ܦͷσʔλϕʔεࣄۀͱ Elasticsearch ೔ᙛ ହี ݕࡧΤϯδχΞ @ Nikkei Inc. 2025/02/26 Elasticsearch Community in Tokyo 2025
2. ೔ܦͷσʔλϕʔεࣄۀ
3. σʔλͰ ݟ௨͢ɺܾΊΔɺಈ͘ɻ త֬ͳҙࢥܾఆΛɻ https://nkbb.nikkei.co.jp/about/
4. ೔ܦͷσʔλϕʔεࣄۀ શίϯςϯπ ݕࡧج൫ (CPF) 200 ࣾҎ্͔Βίϯςϯπఏܞ ɾ ɾ ɾ ɾ ɾ ɾ 12 ͷ৘ใαʔϏε +
5. • ࠃ಺࠷େڃͷϏδωεσʔλϕʔεαʔϏε ◦ 750Ҏ্ͷ৘ใഔମ ◦ 8,600ສࣾͷاۀ৘ใ ◦ 30ສͷਓࣄ৘ใ • ౦ূ1෦্৔اۀͷ7ׂҎ্͕ಋೖ • શจݕࡧͰҙࢥܾఆΛࢧԉ
6. ݕࡧٕज़͕ࢧ͑Δσʔλϕʔεࣄۀ ҙࢥܾఆ ࢧԉ ৴པͰ͖Δ ৘ใݯ ݕࡧٕज़
7. Elasticsearch in Our Search
8. ݕࡧΤϯδϯͷྺ࢙ 1999 2007 2014 2023 May 2024 2018 6 8 7 ΦϯϓϨϛε PanaSearch
9. ΞʔΩςΫνϟ ݕࡧج൫ (CPF) ETL APIs DB 12ͷ৘ใαʔϏε 200 ࣾҎ্͔Βίϯςϯπఏܞ ɾ ɾ ɾ + ɾ ɾ ɾ
10. Elasticsearchͷن໛ײ • ࠷େ 48ϊʔυΫϥελ • 2ԯ υΩϡϝϯτ • +1.5ສ / day • ੑೳ໨ඪ ◦ 200 QPS, ≤200 ms Coordinating Only Nodes (3) Data Nodes (42) Master Nodes (3) ɾ ɾ ɾ
11. ೔ຊ࠷େڃͷElasticsearchಋೖࣄྫ https://www.elastic.co/jp/customers/nikkei-1 https://www.acroquest.co.jp/archives/15788/
12. ࠷ۙͷऔΓ૊Έ: ੜ੒AIؔ࿈
13. 3݄17೔ʹRAGαʔϏεΛϦϦʔε༧ఆ
18. ݱঢ়ͷΞʔΩςΫνϟ ࣭໰ αϒΫΤϦ ݕࡧ݁Ռ هࣄ ճ౴ Gemini Flash text-embedding -3-large Top 30 results Fetches 1000 docs (200 *5) Gemini Pro knn mseach text- embedding-3- large νϟϯΫ
19. ՝୊ײ • ϋΠϒϦουݕࡧͷಋೖ ◦ ݱঢ়ɺϕΫτϧݕࡧͷΈํ͕ਫ਼౓͕ग़͍ͯΔঢ়گ • ϕΫτϧ༰ྔͷංେԽ ◦ 2ԯυΩϡϝϯτΛີϕΫτϧ(1536࣍ݩ)ͰϕΫτϧԽ͢Δͱ3TBҎ্ඞཁ (2.35ഒ) ◦ ೔ຊޠεύʔεϕΫτϧϞσϧ ▪ ElasticͷELSER͸ӳޠʹ࠷దԽ ▪ ೔ຊޠSPLADEϞσϧ͕ొ৔ ◦ BBQ (Better Binary Quantization) ▪ ES 8.17͔Βಋೖ͞ΕͨϕΫτϧѹॖ
20. ·ͱΊ • ೔ܦͷσʔλϕʔεࣄۀ͸200ࣾҎ্͔Βίϯςϯπఏܞɻ ͦͷ๲େͳίϯςϯπΛԣஅݕࡧͰ͖ΔαʔϏεΛల։ • ݕࡧ͸զʑͷࣄۀͷίΞٕज़Ͱ͋Γɺͦͷݕࡧ͸ ElasticsearchͰࢧ͑ΒΕ͍ͯΔ • ੜ੒AIରԠΛੵۃతʹਐΊΔ͜ͱͰɺސ٬ͷҙࢥܾఆʹ͞Β ʹߩݙ
21. We are hiring! https://herp.careers/v1/nikkei/4cJ-o1UA-bz7