Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
28
LADL-Code Mesh V
eramirem
0
200
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
260
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
450
Numerical Analysis for Orbit Propagation
eramirem
0
240
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
390
Other Decks in Science
See All in Science
CV_3_Keypoints
hachama
0
210
白金鉱業Meetup_Vol.20 効果検証ことはじめ / Introduction to Impact Evaluation
brainpadpr
1
1k
コンピュータビジョンによるロボットの視覚と判断:宇宙空間での適応と課題
hf149
1
370
データベース14: B+木 & ハッシュ索引
trycycle
PRO
0
490
Transport information Geometry: Current and Future II
lwc2017
0
210
Symfony Console Facelift
chalasr
2
480
04_石井クンツ昌子_お茶の水女子大学理事_副学長_D_I社会実現へ向けて.pdf
sip3ristex
0
650
会社でMLモデルを作るとは @電気通信大学 データアントレプレナーフェロープログラム
yuto16
1
270
Agent開発フレームワークのOverviewとW&B Weaveとのインテグレーション
siyoo
0
350
academist Prize 4期生 研究トーク延長戦!「美は世界を救う」っていうけど、どうやって?
jimpe_hitsuwari
0
350
データマイニング - グラフ構造の諸指標
trycycle
PRO
0
190
Hakonwa-Quaternion
hiranabe
1
140
Featured
See All Featured
Build your cross-platform service in a week with App Engine
jlugia
232
18k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
45
2.5k
A designer walks into a library…
pauljervisheath
209
24k
Being A Developer After 40
akosma
91
590k
Building Better People: How to give real-time feedback that sticks.
wjessup
369
20k
Building an army of robots
kneath
306
46k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
30
2.7k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
9.7k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
Become a Pro
speakerdeck
PRO
29
5.5k
Writing Fast Ruby
sferik
629
62k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
32
2.3k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture