Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
900
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
19
LADL-Code Mesh V
eramirem
0
190
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
250
The Linear Algebra of Deep Learning
eramirem
2
700
Linear Algebra for FE Developers
eramirem
1
610
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
410
Numerical Analysis for Orbit Propagation
eramirem
0
230
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
Transformers are Universal in Context Learners
gpeyre
0
710
生成AI による論文執筆サポートの手引き(ワークショップ) / A guide to supporting dissertation writing with generative AI (workshop)
ks91
PRO
0
390
機械学習を支える連続最適化
nearme_tech
PRO
1
240
20240420 Global Azure 2024 | Azure Migrate でデータセンターのサーバーを評価&移行してみる
olivia_0707
2
990
How were Quaternion discovered
kinakomoti321
2
1.2k
Reconciling Accuracy, Cost, and Latency of Inference Serving Systems
pjamshidi
0
120
創薬における機械学習技術について
kanojikajino
16
4.9k
Coqで選択公理を形式化してみた
soukouki
0
280
MoveItを使った産業用ロボット向け動作作成方法の紹介 / Introduction to creating motion for industrial robots using MoveIt
ry0_ka
0
290
非同期コミュニケーションの構造 -チャットツールを用いた組織における情報の流れの設計について-
koisono
0
200
白金鉱業Meetup Vol.15 DMLによる条件付処置効果の推定_sotaroIZUMI_20240919
brainpadpr
2
670
サイゼミ用因果推論
lw
1
2.4k
Featured
See All Featured
What's in a price? How to price your products and services
michaelherold
244
12k
StorybookのUI Testing Handbookを読んだ
zakiyama
28
5.5k
Documentation Writing (for coders)
carmenintech
67
4.6k
Java REST API Framework Comparison - PWX 2021
mraible
28
8.4k
Building Applications with DynamoDB
mza
93
6.2k
A better future with KSS
kneath
238
17k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
4
400
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Why You Should Never Use an ORM
jnunemaker
PRO
55
9.2k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
How STYLIGHT went responsive
nonsquared
98
5.3k
Navigating Team Friction
lara
183
15k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture