Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
900
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
19
LADL-Code Mesh V
eramirem
0
180
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
240
The Linear Algebra of Deep Learning
eramirem
2
700
Linear Algebra for FE Developers
eramirem
1
610
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
410
Numerical Analysis for Orbit Propagation
eramirem
0
210
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
ウェーブレットおきもち講座
aikiriao
1
790
山形とさくらんぼに関するレクチャー(YG-900)
07jp27
1
220
240510 COGNAC LabChat
kazh
0
130
AI科学の何が“哲学”の問題になるのか ~問いマッピングの試み~
rmaruy
1
2.3k
WCS-LA-2024
lcolladotor
0
120
Iniciativas independentes de divulgação científica: o caso do Movimento #CiteMulheresNegras
taisso
0
240
白金鉱業Meetup Vol.15 DMLによる条件付処置効果の推定_sotaroIZUMI_20240919
brainpadpr
1
490
ベイズのはなし
techmathproject
0
290
Analysis-Ready Cloud-Optimized Data for your community and the entire world with Pangeo-Forge
jbusecke
0
110
作業領域内の障害物を回避可能なバイナリマニピュレータの設計 / Design of binary manipulator avoiding obstacles in workspace
konakalab
0
160
Causal discovery based on non-Gaussianity and nonlinearity
sshimizu2006
0
190
科学で迫る勝敗の法則(名城大学公開講座.2024年10月) / The principle of victory discovered by science (Open lecture in Meijo Univ. 2024)
konakalab
0
200
Featured
See All Featured
4 Signs Your Business is Dying
shpigford
180
21k
Optimising Largest Contentful Paint
csswizardry
33
2.9k
Intergalactic Javascript Robots from Outer Space
tanoku
269
27k
Documentation Writing (for coders)
carmenintech
65
4.4k
Rails Girls Zürich Keynote
gr2m
94
13k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
506
140k
Statistics for Hackers
jakevdp
796
220k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
27
4.3k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
Being A Developer After 40
akosma
87
590k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture