Share
Embed
Transcript
16 slides
-
ͳͥɺࣅ͍ͯΔผͷϓϩδΣΫτΛݟ͚ͭΔඞཁ͕͋Δ͔ʁ "ϓϩδΣΫτ ɾ໘ੵɿ ᶷ ɾඅ༻ɿԯԁ ɾظؒɿ ɾಛᶃ ݸ͕ࣨଟ͘ɺಛघͳઃඋͱஅ ɾಛᶄ ਖ਼໘ʹಓ࿏͕͋Γɺଞࣼ
#ϓϩδΣΫτ $ϓϩδΣΫτ %ϓϩδΣΫτ ઃඋͷઃஔཁྖ ɾಛੑ֬ೝ ೖࡳʹඞཁͳॻྨɾखଓ͖ɺ ৹ࠪཁྖɾཁ݅Λ֬ೝ ಛघͳܗɾಛੑʹΑΔ ࢿࡐௐୡํ๏Λݕ౼ -
ʮ͍͋·͍͞ʯʹରԠ͢ΔͨΊͷઓུ "ϓϩδΣΫτ ɾ໘ੵɿ ᶷ ɾඅ༻ɿԯԁ ɾظؒɿ ɾྃɿ݄ ɾಛᶃ ݸ͕ࣨଟ͘ɺಛघͳઃඋͱஅ ɾಛᶄ
ਖ਼໘ʹಓ࿏͕͋Γɺଞࣼ ᶃᐆດ͞ΛߏԽ͠ɺ ࣮ʹམͱ͠ࠐΊΔܗ ۩ମԽ ˞Ճॏׂ߹ͷ߹ܭ͕ ʹͳΔΑ͏ʹઃఆͯ͠ݕࡧ ᶄݕࡧ݁ՌෳͷϓϩδΣΫτ͕ ҰͰݟ͑ΔΑ͏ʹఏڙ͢Δ #ϓϩδΣΫτ 50% 25% 25% 0% 0% ໘ੵ අ༻ ಛ ྃ ֊ ɾ໘ੵɿ ᶷ ɾඅ༻ɿԯԁ ɾྃɿ݄ ɾಛ ओཁൖೖܦ࿏͕શͯࣼ $ϓϩδΣΫτ ɾ໘ੵɿ ᶷ ɾඅ༻ɿԯԁ ɾྃɿ݄ ɾಛ ਖ਼໘͔ΒͷԼΓࡔʹҙ %ϓϩδΣΫτ ɾ໘ੵɿ ᶷ ɾඅ༻ɿԯԁ ɾྃɿ݄ ɾಛ ͖͍͢൫ߏʹҙ ྨࣅ ྨࣅ ྨࣅ -
ʮ͍͋·͍ʯͳͱॏΈ͚ͮΛద༻͢ΔGVODUJPO@TDPSF public filterAreaWithScale( origin: number, offset = 10, scale =
1000 ) { this.queryTemplate.query.bool.must.push({ function_score: { functions: [ { exp: { 'document.໘ੵ': { origin: origin, offset: offset, scale: scale, }, }, weight: weight }, ], }, }) return this } %FDBZ'VODUJPOT (BVTTJBO ڑ͕͍ۙ͘͝ͷ͕΄΅ຬɺΕΔ΄Ͳٸݮ &YQPOFOUJBM ڑͷ૿ՃʹͭΕͯҰఆͷׂ߹Ͱܧଓతʹݮগ -JOFBS ڑʹൺྫͯ͠ઢతʹείΞ͕ݮগ -
ʮ͍͋·͍ʯͳͱॏΈ͚ͮΛద༻͢ΔGVODUJPO@TDPSF public filterAreaWithScale( origin: number, offset = 10, scale =
1000 ) { this.queryTemplate.query.bool.must.push({ function_score: { functions: [ { exp: { 'document.໘ੵ': { origin: origin, offset: offset, scale: scale, }, }, weight: weight }, ], }, }) return this } // ʮྨࣅʯΛө͢ΔͨΊͷϝιου // ࣄબޙɺεϥΠμʔͷॳظϩδοΫʹج͍ͮͯ // offset ͱ scale Λө͠·͢ // origin : ൺֱͷج४ͱͳΔ // offset : ج४ͱͷ͕ࠩ͜ͷҎͷ߹ɺείΞ͕ຬʢ1ʣͱΈͳ͞ΕΔൣғ // scale : ج४ͱͷ͕ࠩ scale Λ͑Δ߹ɺείΞʹө͞Εͳ͍ const searchQuery = new SearchQueryHelper() .setPageNumber(1) .setPageSize(25) .filterAreaWithScale(origin, offset, scale) .filterCostWithScale(origin, offset, scale) .filterPeriodWithScale(origin, offset, scale) .filterDeliveryDateWithScale(origin, offset, scale) .setSearchTextVector('ಛ', inputTextEmbedding.vector) -
લॲཧ͕ՄೳͳશͯͷछྨͷσʔλΛΑΓࠐΈɺߏԽ͢Δ | code | name | area | cost |
period | deliveryDate | |------|-----------|------|------------|--------|--------------| | 0001 | A project | 2000 | 1000000000 | 365 | 2023-01-01 | | 0002 | B project | 2000 | 1000000000 | 365 | 2023-01-02 | | 0003 | C project | 2000 | 1000000000 | 365 | 2023-01-03 | root/ ɹᵋ 0001/ ɹ ├ ܖɾਪਐ४උ/ ├ ૯߹ܭըॻ.pdf └ ܖॻ.pdf ├ ٞࣄ/ └ ͦͷଞ ... ϓϩδΣΫτ %# ϓϩδΣΫτ ؔ࿈ࢿྉ ɾॻྨ { "meta": { "code": "0001", "name": "A project" }, "document": { "area": 2000, "cost": 1000000000, "period": 1000000000, "startDate": "2022-01-01", "deliveryDate": 2023-01-01, "type": "GENERAL", "status": "DONE" }, "embeddings": [ { "vector": [ ... ], "text": { "key": "ಛᶃ", "value": "ݸ͕ࣨଟ͘ɺಛघͳઃඋͱஅ..." } }, { "vector": [ ... ], "text": { "key": "ಛᶄ", "value": "ਖ਼໘ʹಓ࿏͕͋Γɺଞࣼ..." } } ] } σʔλͳͲઈରͰදݱͰ͖Δ߲ Ϛελʔ%#͔Β&5-ॲཧ ྨࣅݕࡧʹඞཁͳ ߏԽσʔλੜ -
ϓϩδΣΫτؒͷྨࣅɾಛΛݟ͚ͭΔͨΊͷج४Λཧ root/ ɹᵋ 0001/ ɹ ├ ܖɾਪਐ४උ/ ├ ૯߹ܭըॻ.pdf └
ܖॻ.pdf ├ ٞࣄ/ └ ͦͷଞ ... { "embeddings": [ { "vector": [ ... ], "text": { "key": "ಛᶃ", "value": "ݸ͕ࣨଟ͘ɺಛघͳઃඋͱஅ..." } }, { "vector": [ ... ], "text": { "key": "ಛᶄ", "value": "ਖ਼໘ʹಓ࿏͕͋Γɺଞࣼ..." } } ] } ϓϩδΣΫτ ؔ࿈ ࢿྉɾॻྨ 0$3ରԠͷ7-.ͰಡΈऔΓ ֤߲ʹؔ͢ΔධՁͱαϚϦʔΛ࡞ Ҭɾཧతཁ݅ ɾ൫ɺߦɺؾީ ґཔओʹؔ͢Δࣄ߲ ɾاۀಛੑɺ༷ ֎؍ɾҙঊతಛ ɾ֎นɺృɺ نʹؔ͢Δৄࡉ ɾ༧ࢉɺ໘ੵɺ֊ ઃඋʹؔ͢Δಛ ɾ֤छઃඋɺӄѹࣨ ߏతཁ݅ ɾ๏ɺ໔ɺ -
ࣗવݴޠσʔλͷϋΠϒϦουݕࡧ { embeddings: [ { text: { key: "Ҭɾཧతཁ݅", value:
"..." }, vector: [...] }, { text: { key: "ઃඋʹؔ͢Δಛ", value: "..." }, vector: [...] }, ] } { query: { hybrid: { queries: [ { // ςΩετݕࡧΫΤϦʔ multi_match: { query: SEARCH_TEXT, fields: ["key", "value"] } }, { // ྨࣅݕࡧΫΤϦʔ knn: { embeddings: { vector: [...] } } } ] } }, search_pipeline: { phase_results_processors: [ { 'normalization-processor': { normalization: { technique: "min_max" }, combination: { technique: "arithmetic_mean", parameters: { // είΞϦϯάॏΈ͚ͮ // [text, vector] weights: [0.3, 0.7] } }, ignore_failure: false } } ] } -
σʔλߏԽॲཧߏͷ֓ཁ ϓϩδΣΫτ %# ϓϩδΣΫτ ؔ࿈ ࢿྉɾॻྨ .PEFMT 4UPSBHF &5- 4USVDUVSJOH
0QFO4FBSDI $47࿈ܞ όονॲཧ ϑΝΠϧ Ξοϓϩʔυ όονॲཧ &5-͓Αͼ +40/ߏԽ 0$3 4VNNBSJ[JOH &NCFEEJOH *OEFYJOH ར༻ͨ͠ੜ"*Ϟσϧ ɾamazon.nova-[lite/pro]-* ɾamazon.titan-embedv2-[text/image]-* ɾnanonets/Nanonets-OCR-s (OSS)