Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MeCabとKerasを使ったテキスト分類
Search
masa-ita
February 23, 2019
Technology
1
490
MeCabとKerasを使ったテキスト分類
masa-ita
February 23, 2019
Tweet
Share
More Decks by masa-ita
See All by masa-ita
Ollamaを使ったLocal Language Model活用法
itagakim
1
130
Run Instant NeRF on Docker
itagakim
1
2.3k
3D Clustering and Metric Learning
itagakim
0
330
Cloud TPUの使い方〜BigBirdの日本語学習済みモデルを作る〜
itagakim
0
660
多言語学習済みモデルmT5とは?
itagakim
1
670
AWSのGPUを安く使ってTensorFlowモデルを訓練する方法
itagakim
0
360
最近の自然言語処理モデルの動向
itagakim
1
560
ディープラーニングで芸術はできるか?〜生成系ネットワークの進展〜
itagakim
0
330
AWSとTerraform初心者がやってみたこと
itagakim
1
460
Other Decks in Technology
See All in Technology
DatabricksにOLTPデータベース『Lakebase』がやってきた!
inoutk
0
130
【Oracle Cloud ウェビナー】インフラのプロフェッショナル集団KELが考えるOCIでのソリューション実現
oracle4engineer
PRO
1
100
開発生産性を組織全体の「生産性」へ! 部門間連携の壁を越える実践的ステップ
sudo5in5k
3
7.4k
マネジメントって難しい、けどおもしろい / Management is tough, but fun! #em_findy
ar_tama
7
1.2k
事例で学ぶ!B2B SaaSにおけるSREの実践例/SRE for B2B SaaS: A Real-World Case Study
bitkey
0
120
United Airlines Customer Service– Call 1-833-341-3142 Now!
airhelp
0
170
サイバーエージェントグループのSRE10年の歩みとAI時代の生存戦略
shotatsuge
3
310
20250707-AI活用の個人差を埋めるチームづくり
shnjtk
6
4k
SEQUENCE object comparison - db tech showcase 2025 LT2
nori_shinoda
0
160
関数型プログラミングで 「脳がバグる」を乗り越える
manabeai
2
200
Tokyo_reInforce_2025_recap_iam_access_analyzer
hiashisan
0
190
Operating Operator
shhnjk
1
610
Featured
See All Featured
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Optimizing for Happiness
mojombo
379
70k
A Modern Web Designer's Workflow
chriscoyier
695
190k
Side Projects
sachag
455
42k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
Gamification - CAS2011
davidbonilla
81
5.4k
Facilitating Awesome Meetings
lara
54
6.4k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Transcript
MeCabKeras 2019/2/23 @Python in
3F-*"% Q:<+/M@3F-*8L )9 3F O8L$?.
IDP6S E<6S >16S KFREG6S /M6S C4-*"% 3F-*8L)9 <JNF '0=A#&H ! 5 72; B, ("%
!!$A<7> 7>-=N-Gram .C(2 !$,@ 7>A<A1
0 # $?/<"A<85 3B!$, %&<*'9)+:. %&<*'D46 =;C2E6 0 Ex. MeCab
'!, ",*+$J8 AOIQH=
FORBFO"( E9 RLRB20N16AOIQ H= RLAAG>U &$ CV .@W73 RL?K MS 16E -D16/5:TH= /5:T;=46 )%#+P 46<
livedoor NHN Japan58+- 42 livedoor $' ) #%&* (!*
=. $'1,79 :6;HTML"/<30 https://www.rondhuit.com/download.html#ldcc
livedoor
MeCab
MeCab HN7GSMGegi−69PKPLW`8:%/0-$ &25iGQoegI _@eg1-*,.4'",BC? !.5)(
fdkRm 5'5 V;T[nUJaGoogle Inc. ^p\Ffh]cX +.3-5#><jl = Y ,"5DAbEZ O
MeCab MeCab C++ '& # !*(
Windows %$ https://taku910.github.io/mecab/#download #"+) 32 64 , https://github.com/ikegami-yukino/mecab/releases/tag/v0.996 #"+) Mac %$ Homebrew mecab, mecab-ipadic #!+) Ubuntu %$ apt mecab, mecab-ipadic #!+)
Keras
keras.preprocessing.text.Tokenizer /-.2 /- !%"(8$&5 * #31)76 0)% +4
', fit &5tokenize !%0) %
keras.preprocessing.sequence.pad_sequences ! ( " # $'%
&
BoW: Bag of Words # %EC* G DEC?
- J;/ F<+EC,8=@1/0&%) 58 ()! '"%*$* ,8I209&%) 58 /1 TF-IDF: Term Frequency Inverse Document Frequency EHI2 ><,8 EC:67B4A .1&% )3
Word Embedding a]!.$*2C<@ fTY=!UD :9RPJG5 a]J ?Z10,000 20,000K6
Ni '3&, &.$*2 7<a]![RP7dJ`RPe.$*2 F S< Word Embeddinga]gO Google A; Xb!LWord2vec^V \B W^Ec!80)2H_!LRP IM Word2vec&#(-%1/Qh@Ec!8 )"-1 +4%0)27> Ec!8<@
RNN: Recurrent Neural Network *-H,+.=8 G "!%AB !*DF
@162 ,'/5?)/ G#$&!:(8 RNN> C;79304E LSTMLong Short Term MemoryGRU Gated Recurrent Unit<
BoW DNN
Word EmbeddingGlobalAveragePooling1D
Word EmbeddingRNNLSTM DNN
BoWDNN 0.5E #9("%$)CBoW+/ DNN4: * DBG6GlobalAveragePooling1D1 !$=2F
A LSTM7H2F,- <4: ' ; 7I ?3>8)CLSTM 4: & @:4
NLP,B8?=4-1$!&)%+"C5>@.A 7EFDQ&A-1Sequence-to-Sequence($* Attention :($*.A;3 OpenAIGoogle
Transformer '#Allen Institute 2.ELMo Google G5($*3BERTOpenAI .6GPT-204 <($* 9/