懐かしきCaboChaのレシピをもう一度

at https://tw-meetup.connpass.com/event/330008/

Transcript

1. ջ͔͖͠CaboChaͷϨγϐΛ΋͏Ұ౓ Pr es ented by log5
2. About “log 5 ” ͨͿΜιϑτ΢ΣΞΤϯδχΞ ొஃͱ͔ࠪಡͱ͔࡞ۂ΋͠·͢ ը૾͸X(ID: log 5 )ͷΞΠίϯ
3. ͓඼ॻ͖ CaboChaͱ͸ʁ Πϯετʔϧͯ͠ΈΑ͏ ࢖ͬͯΈΑ͏ ׆༻ͯ͠ΈΑ͏
4. CaboChaͱ͸ʁ ೔ຊޠ܎Γड͚ղੳث จষͷ܎Γड͚ߏ଄Λ෼ੳͯ͘͠ΕΔ Ͳͷ୯ޠ͕Ͳͷ୯ޠʹ܎͍ͬͯΔ͔͕Θ͔Δ
5. “ଠ࿠͸Ֆࢠ͕ಡΜͰ͍ΔຊΛ ࣍࿠ʹ౉ͨ͠”
7. ͱ͜ΖͰɺΈͳ͞Μ͸ ೔ৗͰ͜ΜͳจষΛॻ͍ͯ͠·ͬͨ͜ͱɺͳ͍Ͱ͔͢ʁ
8. “ࢲ͸ܑͱ෕Λܴ͑ʹۭߓʹ޲͔ͬͨɻ”
9. “ࢲ͸ܑͱ෕Λܴ͑ʹۭߓʹ޲͔ͬͨɻ” ʮࢲ͸ܑͱʯ෕Λܴ͑ʹۭߓʹ޲͔ͬͨʁ ࢲ͸ʮܑͱ෕Λʯܴ͑ʹۭߓʹ޲͔ͬͨʁ
11. CaboChaͱ͸ʁ ೔ຊޠ܎Γड͚ղੳث จষͷ܎Γड͚ߏ଄Λ෼ੳͯ͘͠ΕΔ Ͳͷ୯ޠ͕Ͳͷ୯ޠʹ܎͍ͬͯΔ͔͕Θ͔Δ ޡղ͕ੜ·ΕΔ༨஍Λݕग़Ͱ͖Δ…͔΋͠Εͳ͍ Cabochaࣗମʹᐆດੑݕग़ͷػೳ͸ͳ͍͕ɺͦͷ଍͕͔ΓʹͳΔ
12. CaboChaͱ͸ʁ Cabochaࣗମʹᐆດੑݕग़ͷػೳ͸ͳ͍͕ɺͦͷ଍͕͔ΓʹͳΔ ࠷ऴతʹ͜͏͍͏άϥϑΛ࡞ΕΔͱ͍͏఺͕΋͏͍͢͝
13. Πϯετʔϧͯ͠ΈΑ͏ ެࣜαΠτΛಡ΋͏(ಛʹWindows؀ڥͷํ) https: // taku910.github.io/cabocha/ ࣄલʹ CRF ++ ͱ MeCab ΛΠϯετʔϧ͢Δ Chasen Ͱ͸ಈ࡞͠·ͤΜ macOSͳΒ brew install cabocha ͰҰൃ
14. ※͔͜͜Βઌ͸macOSΛϕʔεʹઆ໌͕ਐΈ·͕͢ɺ ଞͷOSͰ΋֓Ͷಉ͡Α͏ʹૢ࡞Ͱ͖Δͱࢥ͍·͢
15. ࢖ͬͯΈΑ͏ ࢼ͠ʹ࢖͏ͳΒλʔϛφϧ͔Β΋ར༻Մೳ λʔϛφϧͰ cabocha ͱೖྗ͢Δͱىಈ จষೖྗ଴ػঢ়ଶʹͳΔͷͰɺ޷͖ͳจষΛೖྗ͢Δ
17. “ࢲ͸ܑͱ෕Λܴ͑ʹۭߓʹ޲͔ͬͨɻ”
21. “෕͕ւ֎ग़ுͰखʹೖΕͨ࣌ܭΛɺ ૆෕͕ݟͨ໨ΛؾʹೖΓɺࢲʹৡͬͯ͘Εͨɻ”
25. “฼͕ҎલۙॴͷΧϑΣͰ஌Γ߹ͬͨళओ͕։࠵͍ͯͨ͠Ξʔτ లͰߪೖͨ͠ֆը͕ɺࢲ͕ඒज़ؗͰݟͨ͜ͱ͕͋ΔըՈͷ࡞඼ ͩͬͨͱޙ͔Β஌Γɺ࡞඼ूΛਤॻؗͰआΓ͖ͯͯ༑ਓʹקΊ Α͏ͱҰ౓ߟ͑ͨ΋ͷͷɺ͢Ͱʹ༑ਓ͕ಉ͡ըՈʹ͍ͭͯݚڀ ͍ͯͨ͜͠ͱΛࠓ೔஌ͬͨɻ”
29. ࢖ͬͯΈΑ͏ ࢼ͠ʹ࢖͏ͳΒλʔϛφϧ͔Β΋ར༻Մೳ λʔϛφϧͰ cabocha ͱೖྗ͢Δͱىಈ จষೖྗ଴ػঢ়ଶʹͳΔͷͰɺ޷͖ͳจষΛೖྗ͢Δ Ҿ਺ͱͯ͠ -f1 Λࢦఆ͢Δͱɺৄࡉ৘ใ͕ಘΒΕΔ
36. ͜ͷग़ྗΛͦͷ··෼ղͯ͠΋͍͍Ͱ͕͢…໘౗ ϓϩάϥϛϯάݴޠͰαοͱ࢖͍͍ͨ Python༻ͷWrapper: cabocha-python Λ࢖ͬͯΈΑ͏ ଞͷݴޠΑΓಋೖɾֶशίετ௿ͦ͏ʢθϩͰ͸ͳ͍ʣ ωοτ্ʹؔ࿈هࣄ΋ଟ͍ҹ৅
37. “cabocha-python” Python Ͱ CaboCha ͕ར༻Ͱ͖Δ ίϚϯυͷग़ྗΛ෼ղ͢ΔͨΊͷॲཧΛࣗ෼Ͱॻ͔ͣʹࡁΉ ܎Γ݁ͼͷؔ܎ΛൺֱతָʹऔΓग़ͤΔ ॳ৺ऀτϥοϓ͋Γ
38. “cabocha-python” ΛΠϯετʔϧ ॳ৺ऀτϥοϓ͋Γ جຊతʹ pip install ͰΠϯετʔϧ͢Δ͜ͱʹͳΔ͕…
41. “cabocha-python” ΛΠϯετʔϧ ॳ৺ऀτϥοϓ͋Γ جຊతʹ pip install ͰΠϯετʔϧ͢Δ͜ͱʹͳΔ͕… 2024 ೥ݱࡏɺ΍ͨΒ෺ʑ͍͠Τϥʔ͕ग़Δ
42. “cabocha-python” ΛΠϯετʔϧ ॳ৺ऀτϥοϓ͋Γ ֤छPath৘ใΛڭ͑ͯ͋͛Δͱղܾ͢Δ…৔߹͕͋Δ
43. “cabocha-python” Λ࢖ͬͯΈΑ͏ Πϯετʔϧ͕ऴΘΕ͹ɺ͋ͱ͸࢖͏͚ͩʂ ͳΜͰ͕͢…
44. “cabocha-python” Λ࢖ͬͯΈΑ͏ Πϯετʔϧ͕ऴΘΕ͹ɺ͋ͱ͸࢖͏͚ͩʂ macOSͷ৔߹ɺApple Silicon Λ౥ࡌ͍ͯ͠Δͱߋʹ᠘͕
45. “cabocha-python” Λ࢖ͬͯΈΑ͏ Πϯετʔϧ͕ऴΘΕ͹ɺ͋ͱ͸࢖͏͚ͩʂ macOSͷ৔߹ɺApple Silicon Λ౥ࡌ͍ͯ͠Δͱߋʹ᠘͕ ↓ͷίʔυ͕࣮ߦ࣌ΤϥʔʹͳΓ·͢
46. ࣮ߦ࣌Τϥʔ (Symbol not found) ੩తղੳͰ͸Τϥʔ͕ग़ͳ͍ͷͰɺૺ۰͢Δͱμϝʔδ͕େ͖͍… ଟ෼ɺΠϯετʔϧ࣌ʹམͱ͖ͯͨ͠ϑΝΠϧ͕ Apple Silicon ͱ஥͕ѱ͍ͷ͕ݪҼ
47. ࣮ߦ࣌Τϥʔ (Symbol not found) ੩తղੳͰ͸Τϥʔ͕ग़ͳ͍ͷͰɺૺ۰͢Δͱμϝʔδ͕େ͖͍… ଟ෼ɺΠϯετʔϧ࣌ʹམͱ͖ͯͨ͠ϑΝΠϧ͕ Apple Silicon ͱ஥͕ѱ͍ͷ͕ݪҼ ιʔείʔυ͔ΒϏϧυ͢Ε͹ղܾ͢Δ…৔߹͕͋Γ·͢ ↓ͷΑ͏ͳखॱͰ࠶Πϯετʔϧ
48. ࠷ॳʹ cabocha Λ࠶Πϯετʔϧ ͨͩ͠ɺιʔείʔυ͔ΒϏϧυ͢Δ ͦͷͨΊ -s (·ͨ͸ -- build-from-source) Λࢦఆ͢Δ
49. ࠷ॳʹ cabocha Λ࠶Πϯετʔϧ ͳͲͱ͍͏ͪΐͬͱා͍͜ͱݴΘΕ·͕͢ɺແࢹɻʢଞʹํ๏ͳ͍΋Μ…ʣ Warning: building from source is not supported! You're on your own. Failures are expected so don't create any issues, please!
50. ࠷ॳʹ cabocha Λ࠶Πϯετʔϧ ࣍ʹ cabocha-python Λ࠶Πϯετʔϧ Apple Silicon Mac ʹద߹͢ΔͨΊ ARCHFLAGS='-arch arm64’ ιʔείʔυ͔ΒόΠφϦΛίϯύΠϧ͢ΔͷͰ -- compile Λ͚ͭΔ
51. “cabocha-python” Λ࢖ͬͯΈΑ͏ ৭ʑ͚͋ͬͨͲɺΠϯετʔϧ͕ऴΘΕ͹ɺ͋ͱ͸࢖͏͚ͩʂ import CaboCha
52. “cabocha-python” Λ࢖ͬͯΈΑ͏ ParserΛ࡞Γɺ෼ղ͢Δ import CaboCha cabocha_parser = CaboCha.Parser() # ղੳثΛ࡞Δ results = cabocha_parser.parse("͜͜ʹ޷͖ͳจষ") # ղੳ͢Δ
53. “cabocha-python” Λ࢖ͬͯΈΑ͏ ParserΛ࡞Γɺ෼ղ͢Δ ͨͩ͠ɺ෼ੳ݁Ռ(͜͜Ͱ͸ results)͸গ͠ѻ͍ͮΒ͍ import CaboCha cabocha_parser = CaboCha.Parser() # ղੳثΛ࡞Δ results = cabocha_parser.parse("͜͜ʹ޷͖ͳจষ") # ղੳ͢Δ
54. “cabocha-python” Λ࢖ͬͯΈΑ͏ ParserΛ࡞Γɺ෼ղ͢Δ ͨͩ͠ɺ෼ੳ݁Ռ(͜͜Ͱ͸ results)͸গ͠ѻ͍ͮΒ͍ [ৄ͍͠ਓ޲͚] Swig ObjectͳͷͰIDEͷิ׬͕͖͔ͳ͍ import CaboCha cabocha_parser = CaboCha.Parser() # ղੳثΛ࡞Δ results = cabocha_parser.parse("͜͜ʹ޷͖ͳจষ") # ղੳ͢Δ
55. “cabocha-python” Λ࢖ͬͯΈΑ͏ [ৄ͍͠ਓ޲͚] Swig ObjectͳͷͰIDEͷิ׬͕͖͔ͳ͍ C++ଆͷϔομΛݟʹߦ͔ͳ͍ͱԿ͕࢖͑Δͷ͔Θ͔Βͳ͍
56. “cabocha-python” Λ࢖ͬͯΈΑ͏ ParserΛ࡞Γɺ෼ղ͢Δ ͨͩ͠ɺ෼ੳ݁Ռ(͜͜Ͱ͸ results)͸গ͠ѻ͍ͮΒ͍ ʮ͜ͷΑ͏ʹॻ͘ͷͩʯͱ͍͏આ໌͕ͦ΋ͧ΋Ϝζ͍ import CaboCha cabocha_parser = CaboCha.Parser() # ղੳثΛ࡞Δ results = cabocha_parser.parse("͜͜ʹ޷͖ͳจষ") # ղੳ͢Δ ԿΛॻ͚͹͍͍ʁʁʁʁʁʁʁʁʁʁʁʁ
57. “cabocha-python” Λ࢖ͬͯΈΑ͏ ParserΛ࡞Γɺ෼ղ͢Δ ͨͩ͠ɺ෼ੳ݁Ռ(͜͜Ͱ͸ results)͸গ͠ѻ͍ͮΒ͍ ʮ͜ͷΑ͏ʹॻ͘ͷͩʯͱ͍͏આ໌͕ͦ΋ͧ΋Ϝζ͍ ͜ͷྲྀΕͰ͍͖ͳΓC++Ӡʑͷઆ໌Λԡ͠෇͚Δͷ͸ແཧ͕͋Ζ͏͔ͱ
58. શ෦טΈࡅ͍ͯઆ໌͢Δ࣌ؒ΋ͳ͍ͷͰ…
59. ௐཧࡁΈͷ΋ͷΛ༻ҙ͠·ͨ͠ https: // github.com/log5r/sample-mecabocha
60. ௐཧࡁΈͷ΋ͷΛ༻ҙ͠·ͨ͠ https: // github.com/log5r/sample-mecabocha main.pyΛ࣮ߦͯ͠ɺจষΛ͍ΕΔ͚ͩ άϥϑ͕ ./out/graph/ ʹग़དྷ·͢
63. ௐཧࡁΈͷ΋ͷΛ༻ҙ͠·ͨ͠ https: // github.com/log5r/sample-mecabocha ΑΖ͚͠Ε͹ࢼͯ͠Έ͍ͯͩ͘͞
64. ௐཧࡁΈͷ΋ͷΛ༻ҙ͠·ͨ͠ https: // github.com/log5r/sample-mecabocha ΑΖ͚͠Ε͹ࢼͯ͠Έ͍ͯͩ͘͞ ϒϨʔΫϙΠϯτషͬͯ؍࡯͢Δ΋Α͠
65. ·ͩ·ͩਚ͖ͳ͍CaboChaͷັྗ ຊ೔঺հͰ͖ͳ͔ͬͨ΋ͷ ղੳϨΠϠͷ࿩ ݻ༗දݱղੳ σʔλʹΑΔֶश ͳͲͳͲ
66. ·ͩ·ͩਚ͖ͳ͍cabochaͷັྗ ຊ೔঺հͰ͖ͳ͔ͬͨ΋ͷ ղੳϨΠϠͷ࿩ ݻ༗දݱղੳ σʔλʹΑΔֶश ͳͲͳͲ ͥͻެࣜαΠτͰʂ https: // taku910.github.io/cabocha/
67. ৯ޙͷײ૝(ؾʹͳΔ఺)
68. CaboChaͷཪଆ Support Vector Machines ʹجͮ͘೔ຊޠ܎Γड͚ղੳث Support Vector Machines ͸ػցֶशΞϧΰϦζϜͷҰͭ ೔ຊޠͷจʹ͓͚Δ୯ޠಉ࢜ͷ܎Γड͚ؔ܎Λ෼ྨɾ༧ଌ
69. CaboChaͷཪଆ ೔ຊޠͷจʹ͓͚Δ୯ޠಉ࢜ͷ܎Γड͚ؔ܎Λ෼ྨɾ༧ଌ ژ౎େֶςΩετίʔύε 4 . 0 ౳Λ༻ֶ͍ͯश
70. CaboChaͷཪଆ ژ౎େֶςΩετίʔύε 4 . 0 ຖ೔৽ฉͷهࣄʹ֤छݴޠ৘ใΛਓखͰ෇༩ 1995 ೥ 1 ݄ 1 ೔͔Β 17 ೔·Ͱͷશهࣄɺ໿ 2 ສจɺ 1 ݄͔Β 12 ݄ ·Ͱͷࣾઆهࣄɺ໿ 2 ສจɺܭ໿ 4 ສจʹରͯ͠ɺܗଶૉɾߏจ ৘ใΛ෇༩
71. CaboChaͷཪଆ ೔ຊޠͷจʹ͓͚Δ୯ޠಉ࢜ͷ܎Γड͚ؔ܎Λ෼ྨɾ༧ଌ ژ౎େֶςΩετίʔύε 4 . 0 ౳Λ༻ֶ͍ͯश ιʔε: “ 1995 ೥ 1 ݄ 1 ೔͔Β 17 ೔·Ͱͷશهࣄ”
72. ؾʹͳΔ͜ͱ ޠኮྗ େৎ෉ʁ
73. ؾʹͳΔ͜ͱ ޠኮྗ ΋ͱ΋ͱݹ͍πʔϧͳͷͰɺ৽ޠͷղੳ͕ෆ҆ఆͳ͜ͱ΋…?
74. “ྩ࿨̒೥ʹ͸ೣϛʔϜ͕όζͬͨ͜ͱ͕͋Δɻ”
78. ؾʹͳΔ͜ͱ ޠኮྗ ͦ΋ͦ΋ݹ͍πʔϧͳͷͰɺ৽ޠͷղੳ͕ෆ҆ఆͳ͜ͱ΋…? ͦ͏͍͑͹ɺCaboCha͸MeCabʹґଘ
79. MeCabͷޠኮྗ ద੾ͳࣙॻΛબͿͱޠኮྗ্͕͕Δ໛༷ JumanDic IPADic NAIST-jdic UniDic Neologd
80. MeCabͷޠኮྗ ద੾ͳࣙॻΛબͿͱޠኮྗ্͕͕Δ໛༷ JumanDic IPADic NAIST-jdic UniDic ←།Ұϝϯςφϯε͞ΕͯΔͬΆ͍ (2023൛͕͋Δ) Neologd
81. ؾʹͳΔ͜ͱ MeCabͷࣙॻΛڧԽ͢Ε͹ɺCaboChaͷ෼ੳਫ਼౓΋্͕Δʁ Θ͔Βͳ͍ CaboChaࣗମ͸܇࿅ࡁΈͷϞσϧʹґଘ͢Δ͸ͣ ͨͱ͑͹ɺMeCabͰ৽͍ࣙ͠ॻΛ࢖༻ͯ͠ɺCaboChaଆͰܴ߹͢Δ Α͏ʹ࠶܇࿅͢ΔͳΒΞϦ…? ৄ͍͠ํͷҙݟΛืूதͰ͢
82. ·ͱΊ CaboCha͸ʢ 2024 ೥Ͱ΋ʣ͍͍ͧ Ұ౓Πϯετʔϧ͢Ε͹ා͘ͳ͍ ܎Γड͚ղੳ͸࣮༻తָ͍ͩ͠͠
83. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ ຊ೔঺հͰ͖ͳ͔ͬͨ΋ͷ ղੳϨΠϠͷ࿩ ݻ༗දݱղੳ σʔλʹΑΔֶश ͳͲͳͲ ͥͻެࣜαΠτͰʂ https: // taku910.github.io/cabocha/