Upgrade to Pro — share decks privately, control downloads, hide ads and more …

日本語の表記ゆれ 解決方法の検討と実装

Takahiko Ito
November 18, 2017
2
2.2k

日本語の表記ゆれ 解決方法の検討と実装

日本語の表記ゆれを解決する方法について検討し実装方法を紹介する。

Takahiko Ito

November 18, 2017
Tweet

More Decks by Takahiko Ito

See All by Takahiko Ito
Elasticsearch における類似度ベクトル検索のベストプラクティスを求めて/es-vector-search
takahiko03
9
6.2k
pfm
takahiko03
0
1.2k
機械学習チームにおけるソフトウェアエンジニア〜役割、キャリア /devsum-2018-summer
takahiko03
8
11k
機械学習プロジェクトを頑健にする施策 ML Ops Study #2
takahiko03
12
4.5k
Cookiecutter Template for Data Scientists Working in Docker Containers
takahiko03
2
2.4k
Cookiecutter for ML experiments with Docker
takahiko03
0
1.1k

Featured

See All Featured
How to train your dragon (web standard)
notwaldorf
91
6k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.2k
Java REST API Framework Comparison - PWX 2021
mraible
30
8.5k
KATA
mclloyd
29
14k
Statistics for Hackers
jakevdp
798
220k
Embracing the Ebb and Flow
colly
85
4.6k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Typedesign – Prime Four
hannesfritz
41
2.6k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
60k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.8k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k

Transcript

  1. ೔ຊޠͷදهΏΕ ղܾํ๏ͷݕ౼ͱ ࣮૷ ҏ౻ܟ඙ 

  2. ࣗݾ঺հ 3FE1FOͱ͍͏จॻͷ νΣοΫπʔϧΛ࡞ͬ ͍ͯ·͢ ݕࡧͱϚΠχϯάΛ΍ Δ͜ͱ͕ଟ͍Ͱ͢ ࢠҭͯத 

  3. ४උɿදهΏΕ 044ϓϩδΣΫτͷυΩϡϝϯτΛॻ͘ͱɺදهΏΕʹ ೰·͞ΕΔ ϰΣτφϜWTϕτφϜ จࣈͷ༳Ε  &YDFMWTΤΫηϧʢจࣈछͷ༳Εʣ ߦ͏WTߦͳ͏ʢૹΓԾ໊ʣ ຊൃදͰ͸ɺදهΏΕ΁ରॲ͢ΔΞΠσΞͱʢ3FE1FO Ͱར༻Ͱ͖Δʣ࣮૷ʹ͍ͭͯ঺հ

    

  4. දهΏΕͱ͍͏໰୊ ଟ͘ͷ৔߹ΫϦςΟΧϧͰ͸ͳ͍ ͔͠͠ɻɻɻൃݟ͢Δͷʹίετ͕͔͔Δɻɻ ෳ਺ਓ਺ͰυΩϡϝϯτΛ੔උɿಛʹදهΏΕ͕ ൃੜ͠΍͍͢ɻɻɻ 

  5. ྫදهΏΕΛؚΉจॻ  ࠷ۙར༻͞Ε͍ͯΔιϑτ΢ΣΞͷதʹ͸ෳ਺ͷܭࢉػ্Ͱಈ࡞ʢ෼ ࢄʣ͢Δ΋ͷ͕ଟ͘ଘࡏ͠·͢ɻ͜ͷΑ͏ͳ෼ࢄιϑτ΢ΣΞ͸ෳ਺ ͷܭࢉػͰಈ࡞͢Δ͜ͱͰେྔͷσʔλΛѻ͑ͨΓɺߴෛՙͳঢ়گʹ ରॲͰ͖ͨΓ͠·͢ɻຊߘͰ͸ ෳ਺ͷܭࢉػʢ$MVTUFSʣͰಈ࡞͢Δ ֤αʔόʔΛʮΠϯελϯεʯͱݺͼ·͢ɻͨͱ͑͹ݕࡧΤϯδϯ΍ ෼ἢσʔλϕʔεͰ͸ΠϯσοΫεΛΫϥελ಺ͷJOTUBODFͰ෼ׂ͠ ͯอ࣋͠·͢ɻ͜ͷΑ͏ͳ৔߹ɺ֤ΠϯσΫεͷ݁ՌΛϚʔδͯ͠Ϋ

    ϥΠΞϯτϓϩάϥϜʹ౉͢ػߏ͕ඞཁͱͳΓ·͢ɻ

  6. ྫදهΏΕΛؚΉจॻ  ࠷ۙར༻͞Ε͍ͯΔιϑτ΢ΣΞͷதʹ͸ෳ਺ͷܭࢉػ্Ͱಈ࡞ʢ෼ ࢄʣ͢Δ΋ͷ͕ଟ͘ଘࡏ͠·͢ɻ͜ͷΑ͏ͳ෼ࢄιϑτ΢ΣΞ͸ෳ਺ ͷܭࢉػͰಈ࡞͢Δ͜ͱͰେྔͷσʔλΛѻ͑ͨΓɺߴෛՙͳঢ়گʹ ରॲͰ͖ͨΓ͠·͢ɻຊߘͰ͸ ෳ਺ͷܭࢉػʢ$MVTUFSʣͰಈ࡞͢Δ ֤αʔόʔΛʮΠϯελϯεʯͱݺͼ·͢ɻͨͱ͑͹ݕࡧΤϯδϯ΍ ෼ἢσʔλϕʔεͰ͸ΠϯσοΫεΛΫϥελ಺ͷJOTUBODFͰ෼ׂ͠ ͯอ࣋͠·͢ɻ͜ͷΑ͏ͳ৔߹ɺ֤ΠϯσΫεͷ݁ՌΛϚʔδͯ͠Ϋ

    ϥΠΞϯτϓϩάϥϜʹ౉͢ػߏ͕ඞཁͱͳΓ·͢ɻ

  7. ݱঢ়·Ͱͷରॲํ๏ 3FE1FOͰ͸ΧλΧφ୯ޠ͚ͩʹ͍ͭͯ͸͋Δఔ ౓ରԠͰ͖Δ จॻ಺͔ΒΧλΧφ୯ޠΛநग़ͯ͠ɺฤूڑ཭ ͕ྨࣅ͢ΔදهΛ࣋ͭ୯ޠϖΞΛநग़ ໰୊఺ɿΧλΧφҎ֎ͷ୯ޠͰ͸ػೳ͠ͳ͍ ɾ Тɾʆ 

  8. ৽͍͠Ξϓϩʔν ୯ޠͷಡΈʹ஫໨ͨ͠ํ๏Λߟ͑ͨ ௐ΂͍ͯͳ͍͕ɺҰൠతͳํ๏͔΋ දهΏΕͷଟ͘͸ಉ͡ಡΈΛ࣋ͭ ྫɿʮ$MVTUFSʯ͸ʮΫϥελʯͱಡΉ Ξϓϩʔνɿಉ͡ಡΈΛ࣋ͭ୯ޠΛؒҧ͍ީิͱ ͯ͠நग़ͯ͋͛͠Δ 

  9. ࣮ࡍͷॲཧ  จॻ಺ͷ֤୯ޠΛಡΈͰΠϯσΫε͢Δ  ಉҰͷಡΈΛ͕࣋ͭผ୯ޠͷϖΞΛΤϥʔͱ͠ ͯग़ྗ͢Δ 

  10. ྫɿΏΒ͗ݕ஌ॲཧ  ෳ਺ͷܭࢉػʢ$MVTUFSʣͰಈ ࡞͢Δ֤αʔόʔΛʮΠϯελ ϯεʯͱݺͼ·͢ɻͨͱ͑͹ݕ ࡧΤϯδϯ΍σʔλϕʔε TFSWFSͰ͸ΠϯσοΫεΛΫ ϥελ಺ͷJOTUBODFͰ෼ׂͯ͠ อ࣋͠·͢ɻ ಡΈ

    ୯ޠϦετ Ϋϥελ $MVTUFSɺΫϥελ ϑΫε΢ ෳ਺ αʔό TFSWFSɺαʔόʔ ʜ ʜ ಉ͡ಡΈΛ࣋ͭ୯ޠϖΞ͸ දهΏΕͷՄೳੑ

  11. ໰୊ಡΈ͸Ͳ͏΍ͬͯऔ ಘ͢Δͷ͔ ղܾ๏ɿࣙॻʹཔΔʂ ೔ຊޠͷܗଶૉղੳثͰར༻͞ΕΔࣙॻʹ͸৭ʑ͋Δ /"*45+%*$ *QBEJD 6OJEJD /&PMPHE 

  12. બ୒ͨࣙ͠ॻɿ/&PMPHE ಡΈ৘ใΛఏڙ͢Δେن ໛ͳࣙॻ /&PMPHE ࣙॻͷαΠζɿ̏̌̌ສ ֤୯ޠͷಡΈ৘ใ͕ఏڙ ͞Ε͍ͯΔ 

  13. ࢖ͬͯΈΔ 3FE1FOWʹ࣮૷ Λ௥Ճ +BQBOFTF&YQSFTTJPO 7BSJBUJPO IUUQSFEQFOIFSPLVBQQDPN 

  14. ࢖ͬͯΈΔʢେ͖Ίॻ੶ʣ աڈʹࣗ਎͕ࣥචͨ͠ຊͷιʔεΛೖྗ Ͳ͏͍ͬͨ݁Ռ͕ಘΒΕΔͷ͔ΛνΣοΫͯ͠Έ ͨɻ 

  15. ݁Ռɿ͏·͍ͬͨ͘Օॴ ਓखͰ͸ؾ͖ͮʹ͍͘දهΏΕ͕औಘͰ͖ͨ ྫɿ ߦͳ͏ ߦ͏ ୹͍ Έ͔͍͡ 

  16. ݁Ռɿ͏·͘ߦ͔ͳ͔ͬͨ Օॴ ಡΈಉ͡Ͱ΋දهΏΕͰ͸ͳ͍ϖΞ΋औಘ͞Εͯ ͠·͏ɻɻɻ l࢓༷ɺ࢖༻ ໊ࢺ  ผه ΂͖ ॿಈࢺ

     දهΏΕͷநग़ػೳͷ࢖༻ʹ͸஫ҙ͕ඞཁ 

  17. ར༻ํ๏ ໰୊఺ɿ໰୊ͷͳ͍ࣄྫ͕நग़͞Εͯ͠·͏ɻ ௨ৗͷνΣοΫͰ͸ར༻ͤͣɺඞཁͳͱ͖͚࣮ͩ ߦ͢ΔͱΑ͛͞ɻ ˠ3FE1FOW͔ΒΤϥʔϨϕϧΛબ୒Ͱ͖ ΔΑ͏ʹͳΓ·ͨ͠ɻ 

  18. 3FE1FOɿΤϥʔϨϕϧ ͷࢦఆ SFEQFODPOGMBOHKB WBMJEBUPST WBMJEBUPSOBNF+BQBOFTF&YQSFTTJPO7BSJBUJPOMFWFM*OGP WBMJEBUPSOBNF%PVCMF/FHBUJWF WBMJEBUPSOBNF1BSBHSBQI/VNCFS WBMJEBUPSOBNF-JTU-FWFM WBMJEBUPST SFEQFODPOG

     ϨϕϧΛ*OGPʹઃఆ͓ͯ͘͠ͱΤϥʔ͸ग़ྗ͞Ε ͳ͍ʢग़ྗΛ͍ͨ͠ͱ͖ʹ͸ύϥϝλύϥϝλlŠ UISFIPMEJOGPzΛࢦఆ͢Δʣɻ

  19. %&.0 SFEQFOUJOGPDDPOGSFEQFODPOGKBYNM TBNQMFEPDKBTBNQMFEPDKBSF 

  20. ·ͱΊ ೔ຊޠͷදهΏΕʹಡΈͰରॲ͢Δํ๏Λ঺հ ࣙॻʢ/&PMPHEʣ͕͋Ε͹ɺ࣮૷͸೉͘͠ͳ͍ ࣗ෼༻ʹ࡞͓ͬͯ͘ͱศར͔΋ 3FE1FOͰͷදهΏΕͷ࢖༻ํ๏͸ͪ͜ΒΛࢀর ͍ͩ͘͞IUUQCJUMZ[08/HJ 

  21. ͝੩ௌ͋Γ͕ͱ͏͝ ͍͟·ͨ͠