Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Deep Learning Book 10その2 / deep learning book 1...
Search
himkt
January 29, 2018
Research
2
160
Deep Learning Book 10その2 / deep learning book 10 vol2
himkt
January 29, 2018
Tweet
Share
More Decks by himkt
See All by himkt
Linformer: paper reading
himkt
0
330
RoBERTa: paper reading
himkt
1
300
NLP SoTA 勉強会 / ner_2019
himkt
2
1.3k
自然言語処理 @ クックパッド / nlp at cookpad
himkt
1
480
Interpretable Machine Learning 6.3 - Prototypes and Criticisms
himkt
2
130
ニューラル固有表現抽出 / Neural Named Entity Recognition
himkt
3
640
ニューラル固有表現抽出器を実装してみる / PyNER
himkt
6
2k
Spacyでお手軽NLP / NLP with spacy
himkt
0
950
ふわふわ系列ラベリング / ner 2018
himkt
5
840
Other Decks in Research
See All in Research
スモールデータ勉強会発表資料
natsutan
0
590
SSII2024 [OS2] 画像、その先へ 〜モーション解析への誘い〜
ssii
PRO
1
1.2k
SSII2024 [OS1] 研究紹介100連発(オープンニング)
ssii
PRO
0
470
SSII2024 [SS1] 拡散モデルの今 〜 2024年の研究動向 〜
ssii
PRO
2
2.1k
[2024.08.30] Gemma-Ko, 오픈 언어모델에 한국어 입히기 @ 머신러닝부트캠프2024
beomi
0
480
20240626_金沢大学_新機能集積回路設計特論_配布用 #makelsi
takasumasakazu
0
140
言語処理学会30周年記念事業留学支援交流会@YANS2024:「学生のための短期留学」
a1da4
1
180
Language is primarily a tool for communication rather than thought
ryou0634
4
630
「Goトレ」のご紹介
smartfukushilab1
0
450
Weekly AI Agents News!
masatoto
22
17k
RCEへの近道
kawakatz
1
750
marukotenant01/tenant-20240826
marketing2024
0
500
Featured
See All Featured
Making Projects Easy
brettharned
113
5.8k
How GitHub (no longer) Works
holman
310
140k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
354
29k
GraphQLの誤解/rethinking-graphql
sonatard
65
9.8k
Creatively Recalculating Your Daily Design Routine
revolveconf
215
12k
Building Your Own Lightsaber
phodgson
101
6k
The Illustrated Children's Guide to Kubernetes
chrisshort
47
48k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
5
480
Optimizing for Happiness
mojombo
375
69k
Atom: Resistance is Futile
akmur
261
25k
VelocityConf: Rendering Performance Case Studies
addyosmani
322
23k
Why You Should Never Use an ORM
jnunemaker
PRO
53
8.9k
Transcript
&DIP4UBUF/FUXPSLT&YQMJDJU.FNPSZ IJNLU!य़ΤϦΞ DEEP LEARNING BOOK 4FRVFODF.PEFMJOH3FDVSSFOUBOE3FDVSTJWF/FUT
&DIP4UBUF/FUXPSLT w 3//ʹֶ͓͍ͯश͕େมͳύϥϝʔλ w ӅΕӅΕ SFDVSSFOUXFJHIUT w ೖྗӅΕ JOQVUXFJHIUT
w &DIP4UBUF/FUXPSL w ӅΕӅΕॏΈΛݻఆ w ֶश͢Δͷʜ w ೖྗӅΕ JOQVUXFJHIUT w ӅΕग़ྗ PVUQVUXFJHIUT
&DIP4UBUF/FUXPSLT IUUQXXXTDIPMBSQFEJBPSHBSUJDMF&DIP@TUBUF@OFUXPSL
,FSOFMNBDIJOFͱͷྨࣅੑ w Χʔωϧ͕ͬͯΔ͜ͱͬͯʁ w ҙͷ͞ͷܥྻΛݻఆͷϕΫτϧࣸ͢ w ݻఆͷϕΫτϧΛ༻͍ͯྨث͕Λղ͘ w ͜ͷܗͷ߹ɼֶशͷج४ͷઃܭ͕༰қͰ͋Δ w
ग़ྗઢܗճؼͷ߹.4&ͰֶशͰ͖Δ w &4/TೖྗΛԿΒ͔ͷϕΫτϧʹࣸ͢ૢ࡞Λ͍ͯ͠Δ w தͷॏΈݻఆ͍ͯ͠Δ ͍͔ʹաڈͷใΛ๛ʹؚΉදݱ͕ಘΒΕΔ ॏΈΛઃఆ͢ΕΑ͍͔ʁ શવҙຯ͕Θ͔Βͣʜ 3//ΛಈతγεςϜͱΈͳ͢ γεςϜ͕҆ఆ͢ΔΑ͏ͳॏΈΛઃఆ͢Δ
-FBLZ6OJUTBOE0UIFS4USBUFHJFTGPS.VMUJQMF5JNF4DBMF w աڈͷใΛ͑ΔͨΊͷ "EEJOH4LJQ$POOFDUJPOTUISPVHI5JNF w ޯͷফࣦͷ͕͘ͳΔ w രൃݩͷ3//ͱಉ͡Ͱൃੜ͢Δ
-FBLZ6OJUTBOEB4QFDUSVNPG%J⒎FSFOU5JNF4DBMFT w աڈͷใΛͲͷఔ͔͢Λ੍ޚ͢Δ 3FNPWJOH$POOFDUJPOT w ͍࣌ࠁͰͷґଘΛ͍࣌ࠁͰͷґଘʹஔ͖͑Δ
-FBLZ6OJUT w աڈͷใΛͲͷ͘Β͍͔͢Λௐ͢Δ w ҠಈฏۉͷΑ͏ͳ;Δ·͍Λ͢Δ w Ћ͕େ͖͍ ʹ͍ۙ աڈͷใΛΑΓอଘ͢Δ w
Ћ͕খ͍͞ ʹ͍ۙ աڈͷใΛ͙͢ʹࣺͯΔ w Ћదʹܾఆ͢ΔϋΠύʔύϥϝʔλ µ(t) ↵µ(t 1) + (1 ↵)v(t)
-POH4IPSU5FSN.FNPSZ w ࣗݾϧʔϓΛಋೖ͢Δ͜ͱͰޯ͕ফ͑ʹ͘͘͢Δ IUUQDPMBIHJUIVCJPQPTUT6OEFSTUBOEJOH-45.T 3// -45.
(BUFE3FDVSSFOU6OJUT w ٙ-45.ෳࡶ͗͢ΔͷͰͳ͍͔ʁ w (36-45.ΑΓߴɾ-45.ͱಉͷੑೳ w ͲͪΒ͕ྑ͍͔λεΫʹΑΔ -45. (36
IUUQTJTBBDDIBOHIBVHJUIVCJP-45.BOE(36'PSNVMB4VNNBSZ
ࣜతʹൺֱ͢ΔʢόΠΞεΛແࢹʣ -45. (36 zt = (xtUz + ht 1Wz)
rt = (xtUr + ht 1Wr) ˜ ht = tanh ⇣ xt + Uh + (rt ht 1)Wh ⌘ ht = (1 zt) ht 1 + zt ˜ ht it = (xtUi + ht 1Wi) ft = (xtUf + ht 1Wf ) ot = (xtUo + ht 1Wg) ˜ Ct = tanh (xtUg + ht 1Wg) Ct = (ft Ct 1 + it ˜ Ct) ht = tanh (Ct) ot (36Ͱೖྗήʔτͱ٫ήʔτ͕౷߹͞Ε͍ͯΔ
0QUJNJ[BUJPOGPS-POH5FSN%FQFOEFODJFT w 3//Λϕʔεͱͨ͠χϡʔϥϧωοτϫʔΫͷඍ w ඇৗʹେ͖ͳΛͱΔPS w ඇৗʹখ͞ͳΛͱΔ w ಛʹɼޯ͕ඇৗʹେ͖ͳͱ͖ʹͲ͏͢Εྑ͍͔ʁ
ޯͷΫϦοϐϯά ޯͷਖ਼نԽ
$MJQQJOH(SBEJFOU w ޯ͕ඇৗʹ େ͖͍cখ͍͞ ͱʁ w ͍͍ͩͨฏΒ͚ͩͲͱ͖Ͳ͖֑͕͋Δ IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM
$MJQQJOH(SBEJFOU w ޯ๏ϕʔεͷख๏ʹΑΔͱʜ w ֑ͷपΓͰ͕ਧ͖ඈΜͰ͠·͏ ޯരൃ w ޯ͕େ͖͘ͳΓ͗ͨ͢ΒޯͷϊϧϜͰׂΔ w
ޯΛHͱͯ͠ʜ w WϋΠύʔύϥϝʔλ ࣗવݴޠॲཧͩͱ͕ଟ͍ g ( gv ||g|| (||g|| > v) g (otherwise)
3FHVMBSJ[JOHUP&ODPVSBHF*OGPSNBUJPO'MPX w ਖ਼ଇԽ߲Λಋೖ͢Δ͜ͱͰʮJOGPSNBUJPOqPXʯΛଅਐ w ͜ͷ߲ͷܭࢉ͍͕͠ɼۙࣅ͕ఏҊ͞Ε͍ͯΔ w $MJQQJOHͱΈ߹ΘͤΔ͜ͱͰهԱͰ͖Δڑ͕৳ͼΔ ⌦ =
X t ⇣||(rh(t) L) @h(t) @h(t 1) || ||(rh(t) L)|| 1 ⌘2
&YQMJDJU.FNPSZ w χϡʔϥϧωοτϫʔΫʜ w ҉తͳใͷอ࣋ಘҙ w ໌ࣔతͳใ ࣄ࣮ ͷอ࣋ۤख w
໌ࣔతͳใΛอ࣋͠ɼਪʹ׆༻͢Δߏ ʢϫʔΩϯάϝϞϦͷಋೖʣ w .FNPSZ/FUXPSLT w /FVSBM5VSJOH.BDIJOF
"TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM
"TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ w ਖ਼֬ͳϝϞϦͷΞυϨεΛग़ྗ͢Δͷ͍͠ w ଟ͘ͷϝϞϦηϧͷॏΈ͖ฏۉΛͱΔ w ॏΈιϑτϚοΫεͳͲͰ࡞Δ ʢͰ͖Δ͚ͩҰՕॴͷϝϞϦΛࢀর͢ΔΑ͏ʹʣ w ϝϞϦηϧεΧϥΑΓϕΫτϧͷํ͕ྑ͍
w ίϯςϯπϕʔεΞυϨογϯά͕ՄೳʹͳΔ w ʮl8FBMMMJWFJOBZFMMPXTVCNBSJOFzΛؚΉՎࢺΛݟ͚ͭΔʯ w ʢϩέʔγϣϯϕʔεΞυϨογϯάͱʁʣ w ʮεϩοτ347ʹ֨ೲ͞Ε͍ͯΔՎࢺΛऔಘ͢Δʯ w ʢΞυϨογϯάΞςϯγϣϯͱಉ͡ܗࣜʣ