Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Deep Learning Book 10その2 / deep learning book 1...
Search
himkt
January 29, 2018
Research
2
190
Deep Learning Book 10その2 / deep learning book 10 vol2
himkt
January 29, 2018
Tweet
Share
More Decks by himkt
See All by himkt
Linformer: paper reading
himkt
0
520
RoBERTa: paper reading
himkt
1
340
NLP SoTA 勉強会 / ner_2019
himkt
2
1.4k
自然言語処理 @ クックパッド / nlp at cookpad
himkt
1
520
Interpretable Machine Learning 6.3 - Prototypes and Criticisms
himkt
2
160
ニューラル固有表現抽出 / Neural Named Entity Recognition
himkt
3
740
ニューラル固有表現抽出器を実装してみる / PyNER
himkt
6
2.1k
Spacyでお手軽NLP / NLP with spacy
himkt
0
1k
ふわふわ系列ラベリング / ner 2018
himkt
5
850
Other Decks in Research
See All in Research
【輪講資料】Moshi: a speech-text foundation model for real-time dialogue
hpprc
3
540
20250605_新交通システム推進議連_熊本都市圏「車1割削減、渋滞半減、公共交通2倍」から考える地方都市交通政策
trafficbrain
0
630
Principled AI ~深層学習時代における課題解決の方法論~
taniai
3
1.2k
集合間Bregmanダイバージェンスと置換不変NNによるその学習
wasyro
0
120
2021年度-基盤研究B-研究計画調書
trycycle
PRO
0
150
20250502_ABEJA_論文読み会_スライド
flatton
0
180
Towards a More Efficient Reasoning LLM: AIMO2 Solution Summary and Introduction to Fast-Math Models
analokmaus
2
670
データサイエンティストの採用に関するアンケート
datascientistsociety
PRO
0
1.1k
20250624_熊本経済同友会6月例会講演
trafficbrain
1
520
AI エージェントを活用した研究再現性の自動定量評価 / scisci2025
upura
1
130
問いを起点に、社会と共鳴する知を育む場へ
matsumoto_r
PRO
0
490
Generative Models 2025
takahashihiroshi
21
13k
Featured
See All Featured
YesSQL, Process and Tooling at Scale
rocio
173
14k
Side Projects
sachag
455
43k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
The Straight Up "How To Draw Better" Workshop
denniskardys
235
140k
Typedesign – Prime Four
hannesfritz
42
2.7k
The Pragmatic Product Professional
lauravandoore
35
6.8k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
Why Our Code Smells
bkeepers
PRO
337
57k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
Transcript
&DIP4UBUF/FUXPSLT&YQMJDJU.FNPSZ IJNLU!य़ΤϦΞ DEEP LEARNING BOOK 4FRVFODF.PEFMJOH3FDVSSFOUBOE3FDVSTJWF/FUT
&DIP4UBUF/FUXPSLT w 3//ʹֶ͓͍ͯश͕େมͳύϥϝʔλ w ӅΕӅΕ SFDVSSFOUXFJHIUT w ೖྗӅΕ JOQVUXFJHIUT
w &DIP4UBUF/FUXPSL w ӅΕӅΕॏΈΛݻఆ w ֶश͢Δͷʜ w ೖྗӅΕ JOQVUXFJHIUT w ӅΕग़ྗ PVUQVUXFJHIUT
&DIP4UBUF/FUXPSLT IUUQXXXTDIPMBSQFEJBPSHBSUJDMF&DIP@TUBUF@OFUXPSL
,FSOFMNBDIJOFͱͷྨࣅੑ w Χʔωϧ͕ͬͯΔ͜ͱͬͯʁ w ҙͷ͞ͷܥྻΛݻఆͷϕΫτϧࣸ͢ w ݻఆͷϕΫτϧΛ༻͍ͯྨث͕Λղ͘ w ͜ͷܗͷ߹ɼֶशͷج४ͷઃܭ͕༰қͰ͋Δ w
ग़ྗઢܗճؼͷ߹.4&ͰֶशͰ͖Δ w &4/TೖྗΛԿΒ͔ͷϕΫτϧʹࣸ͢ૢ࡞Λ͍ͯ͠Δ w தͷॏΈݻఆ͍ͯ͠Δ ͍͔ʹաڈͷใΛ๛ʹؚΉදݱ͕ಘΒΕΔ ॏΈΛઃఆ͢ΕΑ͍͔ʁ શવҙຯ͕Θ͔Βͣʜ 3//ΛಈతγεςϜͱΈͳ͢ γεςϜ͕҆ఆ͢ΔΑ͏ͳॏΈΛઃఆ͢Δ
-FBLZ6OJUTBOE0UIFS4USBUFHJFTGPS.VMUJQMF5JNF4DBMF w աڈͷใΛ͑ΔͨΊͷ "EEJOH4LJQ$POOFDUJPOTUISPVHI5JNF w ޯͷফࣦͷ͕͘ͳΔ w രൃݩͷ3//ͱಉ͡Ͱൃੜ͢Δ
-FBLZ6OJUTBOEB4QFDUSVNPG%J⒎FSFOU5JNF4DBMFT w աڈͷใΛͲͷఔ͔͢Λ੍ޚ͢Δ 3FNPWJOH$POOFDUJPOT w ͍࣌ࠁͰͷґଘΛ͍࣌ࠁͰͷґଘʹஔ͖͑Δ
-FBLZ6OJUT w աڈͷใΛͲͷ͘Β͍͔͢Λௐ͢Δ w ҠಈฏۉͷΑ͏ͳ;Δ·͍Λ͢Δ w Ћ͕େ͖͍ ʹ͍ۙ աڈͷใΛΑΓอଘ͢Δ w
Ћ͕খ͍͞ ʹ͍ۙ աڈͷใΛ͙͢ʹࣺͯΔ w Ћదʹܾఆ͢ΔϋΠύʔύϥϝʔλ µ(t) ↵µ(t 1) + (1 ↵)v(t)
-POH4IPSU5FSN.FNPSZ w ࣗݾϧʔϓΛಋೖ͢Δ͜ͱͰޯ͕ফ͑ʹ͘͘͢Δ IUUQDPMBIHJUIVCJPQPTUT6OEFSTUBOEJOH-45.T 3// -45.
(BUFE3FDVSSFOU6OJUT w ٙ-45.ෳࡶ͗͢ΔͷͰͳ͍͔ʁ w (36-45.ΑΓߴɾ-45.ͱಉͷੑೳ w ͲͪΒ͕ྑ͍͔λεΫʹΑΔ -45. (36
IUUQTJTBBDDIBOHIBVHJUIVCJP-45.BOE(36'PSNVMB4VNNBSZ
ࣜతʹൺֱ͢ΔʢόΠΞεΛແࢹʣ -45. (36 zt = (xtUz + ht 1Wz)
rt = (xtUr + ht 1Wr) ˜ ht = tanh ⇣ xt + Uh + (rt ht 1)Wh ⌘ ht = (1 zt) ht 1 + zt ˜ ht it = (xtUi + ht 1Wi) ft = (xtUf + ht 1Wf ) ot = (xtUo + ht 1Wg) ˜ Ct = tanh (xtUg + ht 1Wg) Ct = (ft Ct 1 + it ˜ Ct) ht = tanh (Ct) ot (36Ͱೖྗήʔτͱ٫ήʔτ͕౷߹͞Ε͍ͯΔ
0QUJNJ[BUJPOGPS-POH5FSN%FQFOEFODJFT w 3//Λϕʔεͱͨ͠χϡʔϥϧωοτϫʔΫͷඍ w ඇৗʹେ͖ͳΛͱΔPS w ඇৗʹখ͞ͳΛͱΔ w ಛʹɼޯ͕ඇৗʹେ͖ͳͱ͖ʹͲ͏͢Εྑ͍͔ʁ
ޯͷΫϦοϐϯά ޯͷਖ਼نԽ
$MJQQJOH(SBEJFOU w ޯ͕ඇৗʹ େ͖͍cখ͍͞ ͱʁ w ͍͍ͩͨฏΒ͚ͩͲͱ͖Ͳ͖֑͕͋Δ IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM
$MJQQJOH(SBEJFOU w ޯ๏ϕʔεͷख๏ʹΑΔͱʜ w ֑ͷपΓͰ͕ਧ͖ඈΜͰ͠·͏ ޯരൃ w ޯ͕େ͖͘ͳΓ͗ͨ͢ΒޯͷϊϧϜͰׂΔ w
ޯΛHͱͯ͠ʜ w WϋΠύʔύϥϝʔλ ࣗવݴޠॲཧͩͱ͕ଟ͍ g ( gv ||g|| (||g|| > v) g (otherwise)
3FHVMBSJ[JOHUP&ODPVSBHF*OGPSNBUJPO'MPX w ਖ਼ଇԽ߲Λಋೖ͢Δ͜ͱͰʮJOGPSNBUJPOqPXʯΛଅਐ w ͜ͷ߲ͷܭࢉ͍͕͠ɼۙࣅ͕ఏҊ͞Ε͍ͯΔ w $MJQQJOHͱΈ߹ΘͤΔ͜ͱͰهԱͰ͖Δڑ͕৳ͼΔ ⌦ =
X t ⇣||(rh(t) L) @h(t) @h(t 1) || ||(rh(t) L)|| 1 ⌘2
&YQMJDJU.FNPSZ w χϡʔϥϧωοτϫʔΫʜ w ҉తͳใͷอ࣋ಘҙ w ໌ࣔతͳใ ࣄ࣮ ͷอ࣋ۤख w
໌ࣔతͳใΛอ࣋͠ɼਪʹ׆༻͢Δߏ ʢϫʔΩϯάϝϞϦͷಋೖʣ w .FNPSZ/FUXPSLT w /FVSBM5VSJOH.BDIJOF
"TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM
"TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ w ਖ਼֬ͳϝϞϦͷΞυϨεΛग़ྗ͢Δͷ͍͠ w ଟ͘ͷϝϞϦηϧͷॏΈ͖ฏۉΛͱΔ w ॏΈιϑτϚοΫεͳͲͰ࡞Δ ʢͰ͖Δ͚ͩҰՕॴͷϝϞϦΛࢀর͢ΔΑ͏ʹʣ w ϝϞϦηϧεΧϥΑΓϕΫτϧͷํ͕ྑ͍
w ίϯςϯπϕʔεΞυϨογϯά͕ՄೳʹͳΔ w ʮl8FBMMMJWFJOBZFMMPXTVCNBSJOFzΛؚΉՎࢺΛݟ͚ͭΔʯ w ʢϩέʔγϣϯϕʔεΞυϨογϯάͱʁʣ w ʮεϩοτ347ʹ֨ೲ͞Ε͍ͯΔՎࢺΛऔಘ͢Δʯ w ʢΞυϨογϯάΞςϯγϣϯͱಉ͡ܗࣜʣ