Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DeepNLP_BackPropagation_Rnn_and_Cnn
Search
izuna385
July 02, 2018
Science
0
160
DeepNLP_BackPropagation_Rnn_and_Cnn
深層学習による自然言語処理 2.5から2.9まで
izuna385
July 02, 2018
Tweet
Share
More Decks by izuna385
See All by izuna385
jel: japanese entity linker
izuna385
0
340
Firebase-React-App
izuna385
0
240
React+FastAPIを用いた簡単なWebアプリ作製
izuna385
0
1.6k
UseCase of Entity Linking
izuna385
0
550
Unofficial slides: From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains (ACL 2020)
izuna385
1
650
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
izuna385
0
840
Zero-shot Entity Linking with Dense Entity Retrieval (Unofficial slides) and Entity Linking future directions
izuna385
3
1k
Entity representation with relational attention
izuna385
0
80
Zero-Shot Entity Linking by Reading Entity Descriptions
izuna385
0
540
Other Decks in Science
See All in Science
統計学入門講座 第1回スライド
techmathproject
0
290
Iniciativas independentes de divulgação científica: o caso do Movimento #CiteMulheresNegras
taisso
0
1.3k
Valuable Lessons Learned on Kaggle’s ARC AGI LLM Challenge (PyDataGlobal 2024)
ianozsvald
0
280
山形とさくらんぼに関するレクチャー(YG-900)
07jp27
1
290
Tensor Representations in Signal Processing and Machine Learning (Tutorial at APSIPA-ASC 2020)
yokotatsuya
0
170
システム数理と応用分野の未来を切り拓くロードマップ・エンターテインメント(スポーツ)への応用 / Applied mathematics for sports entertainment
konakalab
1
260
Machine Learning for Materials (Challenge)
aronwalsh
0
260
トラブルがあったコンペに学ぶデータ分析
tereka114
2
1.5k
私たちのプロダクトにとってのよいテスト/good test for our products
camel_404
0
280
ACL読み会2024@名大 REANO: Optimising Retrieval-Augmented Reader Models through Knowledge Graph Generation
takuma_matsubara
0
180
メール送信サーバの集約における透過型SMTP プロキシの定量評価 / Quantitative Evaluation of Transparent SMTP Proxy in Email Sending Server Aggregation
linyows
0
840
大規模言語モデルの論理構造の把握能力と予測モデルの生成
fuyu_quant0
0
120
Featured
See All Featured
Rails Girls Zürich Keynote
gr2m
94
13k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
30k
How GitHub (no longer) Works
holman
314
140k
A Tale of Four Properties
chriscoyier
158
23k
Gamification - CAS2011
davidbonilla
81
5.2k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Code Review Best Practice
trishagee
67
18k
It's Worth the Effort
3n
184
28k
Unsuck your backbone
ammeep
670
57k
BBQ
matthewcrist
88
9.6k
Optimizing for Happiness
mojombo
377
70k
Transcript
5 . 1 2
• : D !(#) 1 ∇!(#) • L 1 )
) ( 1 S G G 1 & '()*+, = .(/ 012 .(⋯ .(/ 2 '()*+, + 5(2)))) 62 67 68 69 /(:) ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ;2 ;7 /(:<2) = − 1 = = + 1
. . ! ℎ($) ℎ(&) ℎ(') (($) ((&) ((') ℎ(')=
( ' ( & ( $ ! *+(,) *- = *+(,) *.(/) 0 *.(/) *.(1) 0 *.(1) *- •
. . ! ℎ($) ℎ(&) '($) '(&)
. .
None
(
.( )
( )
2'+ )NN 2 #) 0%&1&-" (3*, )
.( !/$→ → Residual Connection, Batch Nomalization( ) ! Loss func ! Loss func
: Residual Connection –– F(x) (→-!()) F(x) + x
→ & " - Identity Mapping ' : -*&%,+' ./$ # Identity – [1] He, Kaiming, et al. "Identity mappings in deep residual networks." European Conference on Computer Vision. Springer, Cham, 2016. . .
. . (2.33) (2.34)
0 1 . 2 C2 2 " 3 2 3
2 ) 2 2 ( 3 23 2 !"#$ !" %"&$ %"#$ %" %"&$ '() '() '() '*+, '*+, -"#$ = /(!"#$ ) -" -"&$ %" !" M I NR 2 '*+, O L '() : input P / L !" = '*+, 2 !"#$ + '() %" ,, L
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )( • 23 !* # 1
: !"#$ !%& '( )( … … )*
'* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/
. - •
) (
8 99 2 :9.8 9 9 6 5 3 2
2 5 2 28 79 8 3 9 1 56 2 59 7 /0-
-5 1 02 1 25 58 8 ./ 8 .2
0 8
22/1 444 1 1 - 2 3--.-.3-. 11
http://deeplearning.stanford.edu/wiki/index .php/Feature_extraction_using_convolution
/885 6 0: 6. 5 5
RNN Vanishing/Exploding Gradient : !"#$ !%&
'( )( … … )* '* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/