Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
28
LADL-Code Mesh V
eramirem
0
200
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
260
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
440
Numerical Analysis for Orbit Propagation
eramirem
0
240
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
390
Other Decks in Science
See All in Science
データベース11: 正規化(1/2) - 望ましくない関係スキーマ
trycycle
PRO
0
930
baseballrによるMLBデータの抽出と階層ベイズモデルによる打率の推定 / TokyoR118
dropout009
1
550
ウェブ・ソーシャルメディア論文読み会 第25回: Differences in misinformation sharing can lead to politically asymmetric sanctions (Nature, 2024)
hkefka385
0
130
機械学習 - 授業概要
trycycle
PRO
0
230
データマイニング - コミュニティ発見
trycycle
PRO
0
140
3次元点群を利用した植物の葉の自動セグメンテーションについて
kentaitakura
2
1.3k
データベース03: 関係データモデル
trycycle
PRO
1
250
サイゼミ用因果推論
lw
1
7.5k
データベース04: SQL (1/3) 単純質問 & 集約演算
trycycle
PRO
0
980
データベース08: 実体関連モデルとは?
trycycle
PRO
0
930
07_浮世満理子_アイディア高等学院学院長_一般社団法人全国心理業連合会代表理事_紹介資料.pdf
sip3ristex
0
590
Explanatory material
yuki1986
0
390
Featured
See All Featured
A designer walks into a library…
pauljervisheath
207
24k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
50
5.5k
4 Signs Your Business is Dying
shpigford
184
22k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Navigating Team Friction
lara
189
15k
RailsConf 2023
tenderlove
30
1.2k
The Language of Interfaces
destraynor
160
25k
The World Runs on Bad Software
bkeepers
PRO
70
11k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
A Modern Web Designer's Workflow
chriscoyier
695
190k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture