Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Deep Learning
Search
Abhinav Tushar
September 10, 2015
Research
6
260
Deep Learning
Introductory talk on deep learning
Abhinav Tushar
September 10, 2015
Tweet
Share
More Decks by Abhinav Tushar
See All by Abhinav Tushar
the garden of eden
lepisma
0
85
Technology
lepisma
0
67
Bio-Inspired Computing
lepisma
0
81
Maestro
lepisma
0
100
War and Economics
lepisma
0
100
Other Decks in Research
See All in Research
Weekly AI Agents News! 7月号 論文のアーカイブ
masatoto
1
230
システムから変える 自分と世界を変えるシステムチェンジの方法論 / Systems Change Approaches
dmattsun
3
870
[CV勉強会@関東 CVPR2024] Visual Layout Composer: Image-Vector Dual Diffusion Model for Design Layout Generation / kantocv 61th CVPR 2024
shunk031
1
460
湯村研究室の紹介2024 / yumulab2024
yumulab
0
280
Language is primarily a tool for communication rather than thought
ryou0634
4
740
テキストマイニングことはじめー基本的な考え方からメディアディスコース研究への応用まで
langstat
1
120
20240918 交通くまもとーく 未来の鉄道網編(こねくま)
trafficbrain
0
230
クロスセクター効果研究会 熊本都市交通リノベーション~「車1割削減、渋滞半減、公共交通2倍」の実現へ~
trafficbrain
0
260
言語と数理の交差点:テキストの埋め込みと構造のモデル化 (IBIS 2024 チュートリアル)
yukiar
3
740
KDD論文読み会2024: False Positive in A/B Tests
ryotoitoi
0
200
外積やロドリゲスの回転公式を利用した点群の回転
kentaitakura
1
650
新規のC言語処理系を実装することによる 組込みシステム研究にもたらす価値 についての考察
zacky1972
0
140
Featured
See All Featured
[RailsConf 2023] Rails as a piece of cake
palkan
52
4.9k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
329
21k
Teambox: Starting and Learning
jrom
133
8.8k
Speed Design
sergeychernyshev
25
620
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
250
21k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
159
15k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
StorybookのUI Testing Handbookを読んだ
zakiyama
27
5.3k
What's new in Ruby 2.0
geeforr
343
31k
Automating Front-end Workflow
addyosmani
1366
200k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
26
1.4k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
1.9k
Transcript
D E E P L E A R N I
N G
models AE / SAE RBM / DBN CNN RNN /
LSTM Memnet / NTM agenda questions What ? Why ? How ? Next ?
what why how next What ? AI technique for learning
multiple levels of abstractions directly from raw information
what why how next Primitive rule based AI Tailored systems
Hand Crafted Program Output Input
what why how next Classical machine learning Learning from custom
features Hand Crafted Features Learning System Output Input
what why how next Deep Learning based AI Learn everything
Learned Features (Lower Level) Learned Features (Higher Level) Learning System Output Input
None
https://www.youtube.com/watch?v=Q70ulPJW3Gk PPTX PDF (link to video below)
With the capacity to represent the world in signs and
symbols, comes the capacity to change it Elizabeth Kolbert (The Sixth Extinction) “
Why The buzz ?
what why how next Google Trends Deep Learning
what why how next
Crude timeline of Neural Networks 1950 1980 1990 2000 Perceptron
Backprop & Application NN Winter
2010 Stacking RBMs Deep Learning fuss
HUGE DATA Large Synoptic Survey Telescope (2022) 30 TB/night
HUGE CAPABILITIES GPGPU ~20x speedup Powerful Clusters
HUGE SUCCESS Speech, text understanding Robotics / Computer Vision Business
/ Big Data Artificial General Intelligence (AGI)
How its done ?
what why how next Shallow Network ℎ ℎ = (,
0) = ′(ℎ, 1) = (, ) minimize
what why how next Deep Network
what why how next Deep Network More abstract features Stellar
performance Vanishing Gradient Overfitting
what why how next Autoencoder ℎ Unsupervised Feature Learning
what why how next Stacked Autoencoder Y. Bengio et. all;
Greedy Layer-Wise Training of Deep Networks
what why how next Stacked Autoencoder 1. Unsupervised, layer by
layer pretraining 2. Supervised fine tuning
what why how next Deep Belief Network 2006 breakthrough Stacking
Restricted Boltzmann Machines (RBMs) Hinton, G. E., Osindero, S. and Teh, Y.; A fast learning algorithm for deep belief nets
Rethinking Computer Vision
what why how next Traditional Image Classification pipeline Feature Extraction
(SIFT, SURF etc.) Classifier (SVM, NN etc.)
what why how next Convolutional Neural Network Images taken from
deeplearning.net
what why how next Convolutional Neural Network
what why how next Convolutional Neural Network Images taken from
deeplearning.net
what why how next Convolutional Neural Network
what why how next The Starry Night Vincent van Gogh
Leon A. Gatys, Alexander S. Ecker and Matthias Bethge; A Neural Algorithm of Artistic Style
what why how next
what why how next Scene Description CNN + RNN Oriol
Vinyals et. all; Show and Tell: A Neural Image Caption Generator
Learning Sequences
what why how next Recurrent Neural Network Simple Elman Version
ℎ ℎ = ( , ℎ−1 , 0, 1) = ′(ℎ , 2)
what why how next Long Short Term Memory (LSTM) add
memory cells learn access mechanism Sepp Hochreiter and Jürgen Schmidhuber; Long short-term memory
None
what why how next
what why how next Fooling Deep Networks Anh Nguyen, Jason
Yosinski, Jeff Clune; Deep Neural Networks are Easily Fooled
Next Cool things to try
what why how next Hyperparameter optimization bayesian Optimization methods adadelta,
rmsprop . . . Regularization dropout, dither . . .
what why how next Attention & Memory NTMs, Memory Networks,
Stack RNNs . . . NLP Translation, description
what why how next Cognitive Hardware FPGA, GPU, Neuromorphic Chips
Scalable DL map-reduce, compute clusters
what why how next Deep Reinforcement Learning deepmindish things, deep
Q learning Energy models RBMs, DBNs . . .
https://www.reddit.com/r/MachineLearning/wiki
Theano (Python) | Torch (lua) | Caffe (C++) Github is
a friend
@AbhinavTushar ?