Upgrade to Pro — share decks privately, control downloads, hide ads and more …

日本語の表記ゆれ 解決方法の検討と実装

Takahiko Ito
November 18, 2017
2
2.1k

日本語の表記ゆれ 解決方法の検討と実装

日本語の表記ゆれを解決する方法について検討し実装方法を紹介する。

Takahiko Ito

November 18, 2017
Tweet

More Decks by Takahiko Ito

See All by Takahiko Ito
Elasticsearch における類似度ベクトル検索のベストプラクティスを求めて/es-vector-search
takahiko03
9
6.1k
pfm
takahiko03
0
1.1k
機械学習チームにおけるソフトウェアエンジニア〜役割、キャリア /devsum-2018-summer
takahiko03
8
11k
機械学習プロジェクトを頑健にする施策 ML Ops Study #2
takahiko03
12
4.5k
Cookiecutter Template for Data Scientists Working in Docker Containers
takahiko03
2
2.3k
Cookiecutter for ML experiments with Docker
takahiko03
0
1.1k

Featured

See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
73
5.4k
It's Worth the Effort
3n
183
28k
Building a Scalable Design System with Sketch
lauravandoore
460
33k
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
Producing Creativity
orderedlist
PRO
341
39k
GraphQLとの向き合い方2022年版
quramy
44
13k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
48
2.2k
A Tale of Four Properties
chriscoyier
157
23k
Why Our Code Smells
bkeepers
PRO
335
57k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
28
2.1k
Bootstrapping a Software Product
garrettdimon
PRO
305
110k

Transcript

  1. ೔ຊޠͷදهΏΕ ղܾํ๏ͷݕ౼ͱ ࣮૷ ҏ౻ܟ඙ 

  2. ࣗݾ঺հ 3FE1FOͱ͍͏จॻͷ νΣοΫπʔϧΛ࡞ͬ ͍ͯ·͢ ݕࡧͱϚΠχϯάΛ΍ Δ͜ͱ͕ଟ͍Ͱ͢ ࢠҭͯத 

  3. ४උɿදهΏΕ 044ϓϩδΣΫτͷυΩϡϝϯτΛॻ͘ͱɺදهΏΕʹ ೰·͞ΕΔ ϰΣτφϜWTϕτφϜ จࣈͷ༳Ε  &YDFMWTΤΫηϧʢจࣈछͷ༳Εʣ ߦ͏WTߦͳ͏ʢૹΓԾ໊ʣ ຊൃදͰ͸ɺදهΏΕ΁ରॲ͢ΔΞΠσΞͱʢ3FE1FO Ͱར༻Ͱ͖Δʣ࣮૷ʹ͍ͭͯ঺հ

    

  4. දهΏΕͱ͍͏໰୊ ଟ͘ͷ৔߹ΫϦςΟΧϧͰ͸ͳ͍ ͔͠͠ɻɻɻൃݟ͢Δͷʹίετ͕͔͔Δɻɻ ෳ਺ਓ਺ͰυΩϡϝϯτΛ੔උɿಛʹදهΏΕ͕ ൃੜ͠΍͍͢ɻɻɻ 

  5. ྫදهΏΕΛؚΉจॻ  ࠷ۙར༻͞Ε͍ͯΔιϑτ΢ΣΞͷதʹ͸ෳ਺ͷܭࢉػ্Ͱಈ࡞ʢ෼ ࢄʣ͢Δ΋ͷ͕ଟ͘ଘࡏ͠·͢ɻ͜ͷΑ͏ͳ෼ࢄιϑτ΢ΣΞ͸ෳ਺ ͷܭࢉػͰಈ࡞͢Δ͜ͱͰେྔͷσʔλΛѻ͑ͨΓɺߴෛՙͳঢ়گʹ ରॲͰ͖ͨΓ͠·͢ɻຊߘͰ͸ ෳ਺ͷܭࢉػʢ$MVTUFSʣͰಈ࡞͢Δ ֤αʔόʔΛʮΠϯελϯεʯͱݺͼ·͢ɻͨͱ͑͹ݕࡧΤϯδϯ΍ ෼ἢσʔλϕʔεͰ͸ΠϯσοΫεΛΫϥελ಺ͷJOTUBODFͰ෼ׂ͠ ͯอ࣋͠·͢ɻ͜ͷΑ͏ͳ৔߹ɺ֤ΠϯσΫεͷ݁ՌΛϚʔδͯ͠Ϋ

    ϥΠΞϯτϓϩάϥϜʹ౉͢ػߏ͕ඞཁͱͳΓ·͢ɻ

  6. ྫදهΏΕΛؚΉจॻ  ࠷ۙར༻͞Ε͍ͯΔιϑτ΢ΣΞͷதʹ͸ෳ਺ͷܭࢉػ্Ͱಈ࡞ʢ෼ ࢄʣ͢Δ΋ͷ͕ଟ͘ଘࡏ͠·͢ɻ͜ͷΑ͏ͳ෼ࢄιϑτ΢ΣΞ͸ෳ਺ ͷܭࢉػͰಈ࡞͢Δ͜ͱͰେྔͷσʔλΛѻ͑ͨΓɺߴෛՙͳঢ়گʹ ରॲͰ͖ͨΓ͠·͢ɻຊߘͰ͸ ෳ਺ͷܭࢉػʢ$MVTUFSʣͰಈ࡞͢Δ ֤αʔόʔΛʮΠϯελϯεʯͱݺͼ·͢ɻͨͱ͑͹ݕࡧΤϯδϯ΍ ෼ἢσʔλϕʔεͰ͸ΠϯσοΫεΛΫϥελ಺ͷJOTUBODFͰ෼ׂ͠ ͯอ࣋͠·͢ɻ͜ͷΑ͏ͳ৔߹ɺ֤ΠϯσΫεͷ݁ՌΛϚʔδͯ͠Ϋ

    ϥΠΞϯτϓϩάϥϜʹ౉͢ػߏ͕ඞཁͱͳΓ·͢ɻ

  7. ݱঢ়·Ͱͷରॲํ๏ 3FE1FOͰ͸ΧλΧφ୯ޠ͚ͩʹ͍ͭͯ͸͋Δఔ ౓ରԠͰ͖Δ จॻ಺͔ΒΧλΧφ୯ޠΛநग़ͯ͠ɺฤूڑ཭ ͕ྨࣅ͢ΔදهΛ࣋ͭ୯ޠϖΞΛநग़ ໰୊఺ɿΧλΧφҎ֎ͷ୯ޠͰ͸ػೳ͠ͳ͍ ɾ Тɾʆ 

  8. ৽͍͠Ξϓϩʔν ୯ޠͷಡΈʹ஫໨ͨ͠ํ๏Λߟ͑ͨ ௐ΂͍ͯͳ͍͕ɺҰൠతͳํ๏͔΋ දهΏΕͷଟ͘͸ಉ͡ಡΈΛ࣋ͭ ྫɿʮ$MVTUFSʯ͸ʮΫϥελʯͱಡΉ Ξϓϩʔνɿಉ͡ಡΈΛ࣋ͭ୯ޠΛؒҧ͍ީิͱ ͯ͠நग़ͯ͋͛͠Δ 

  9. ࣮ࡍͷॲཧ  จॻ಺ͷ֤୯ޠΛಡΈͰΠϯσΫε͢Δ  ಉҰͷಡΈΛ͕࣋ͭผ୯ޠͷϖΞΛΤϥʔͱ͠ ͯग़ྗ͢Δ 

  10. ྫɿΏΒ͗ݕ஌ॲཧ  ෳ਺ͷܭࢉػʢ$MVTUFSʣͰಈ ࡞͢Δ֤αʔόʔΛʮΠϯελ ϯεʯͱݺͼ·͢ɻͨͱ͑͹ݕ ࡧΤϯδϯ΍σʔλϕʔε TFSWFSͰ͸ΠϯσοΫεΛΫ ϥελ಺ͷJOTUBODFͰ෼ׂͯ͠ อ࣋͠·͢ɻ ಡΈ

    ୯ޠϦετ Ϋϥελ $MVTUFSɺΫϥελ ϑΫε΢ ෳ਺ αʔό TFSWFSɺαʔόʔ ʜ ʜ ಉ͡ಡΈΛ࣋ͭ୯ޠϖΞ͸ දهΏΕͷՄೳੑ

  11. ໰୊ಡΈ͸Ͳ͏΍ͬͯऔ ಘ͢Δͷ͔ ղܾ๏ɿࣙॻʹཔΔʂ ೔ຊޠͷܗଶૉղੳثͰར༻͞ΕΔࣙॻʹ͸৭ʑ͋Δ /"*45+%*$ *QBEJD 6OJEJD /&PMPHE 

  12. બ୒ͨࣙ͠ॻɿ/&PMPHE ಡΈ৘ใΛఏڙ͢Δେن ໛ͳࣙॻ /&PMPHE ࣙॻͷαΠζɿ̏̌̌ສ ֤୯ޠͷಡΈ৘ใ͕ఏڙ ͞Ε͍ͯΔ 

  13. ࢖ͬͯΈΔ 3FE1FOWʹ࣮૷ Λ௥Ճ +BQBOFTF&YQSFTTJPO 7BSJBUJPO IUUQSFEQFOIFSPLVBQQDPN 

  14. ࢖ͬͯΈΔʢେ͖Ίॻ੶ʣ աڈʹࣗ਎͕ࣥචͨ͠ຊͷιʔεΛೖྗ Ͳ͏͍ͬͨ݁Ռ͕ಘΒΕΔͷ͔ΛνΣοΫͯ͠Έ ͨɻ 

  15. ݁Ռɿ͏·͍ͬͨ͘Օॴ ਓखͰ͸ؾ͖ͮʹ͍͘දهΏΕ͕औಘͰ͖ͨ ྫɿ ߦͳ͏ ߦ͏ ୹͍ Έ͔͍͡ 

  16. ݁Ռɿ͏·͘ߦ͔ͳ͔ͬͨ Օॴ ಡΈಉ͡Ͱ΋දهΏΕͰ͸ͳ͍ϖΞ΋औಘ͞Εͯ ͠·͏ɻɻɻ l࢓༷ɺ࢖༻ ໊ࢺ  ผه ΂͖ ॿಈࢺ

     දهΏΕͷநग़ػೳͷ࢖༻ʹ͸஫ҙ͕ඞཁ 

  17. ར༻ํ๏ ໰୊఺ɿ໰୊ͷͳ͍ࣄྫ͕நग़͞Εͯ͠·͏ɻ ௨ৗͷνΣοΫͰ͸ར༻ͤͣɺඞཁͳͱ͖͚࣮ͩ ߦ͢ΔͱΑ͛͞ɻ ˠ3FE1FOW͔ΒΤϥʔϨϕϧΛબ୒Ͱ͖ ΔΑ͏ʹͳΓ·ͨ͠ɻ 

  18. 3FE1FOɿΤϥʔϨϕϧ ͷࢦఆ SFEQFODPOGMBOHKB WBMJEBUPST WBMJEBUPSOBNF+BQBOFTF&YQSFTTJPO7BSJBUJPOMFWFM*OGP WBMJEBUPSOBNF%PVCMF/FHBUJWF WBMJEBUPSOBNF1BSBHSBQI/VNCFS WBMJEBUPSOBNF-JTU-FWFM WBMJEBUPST SFEQFODPOG

     ϨϕϧΛ*OGPʹઃఆ͓ͯ͘͠ͱΤϥʔ͸ग़ྗ͞Ε ͳ͍ʢग़ྗΛ͍ͨ͠ͱ͖ʹ͸ύϥϝλύϥϝλlŠ UISFIPMEJOGPzΛࢦఆ͢Δʣɻ

  19. %&.0 SFEQFOUJOGPDDPOGSFEQFODPOGKBYNM TBNQMFEPDKBTBNQMFEPDKBSF 

  20. ·ͱΊ ೔ຊޠͷදهΏΕʹಡΈͰରॲ͢Δํ๏Λ঺հ ࣙॻʢ/&PMPHEʣ͕͋Ε͹ɺ࣮૷͸೉͘͠ͳ͍ ࣗ෼༻ʹ࡞͓ͬͯ͘ͱศར͔΋ 3FE1FOͰͷදهΏΕͷ࢖༻ํ๏͸ͪ͜ΒΛࢀর ͍ͩ͘͞IUUQCJUMZ[08/HJ 

  21. ͝੩ௌ͋Γ͕ͱ͏͝ ͍͟·ͨ͠