Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
FlexiBO: A Decoupled Cost-Aware Multi-Objective...
Search
Pooyan Jamshidi
February 29, 2024
Science
0
110
FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization of Deep Neural Networks
AAAI 2024
Pooyan Jamshidi
February 29, 2024
Tweet
Share
More Decks by Pooyan Jamshidi
See All by Pooyan Jamshidi
Reconciling Accuracy, Cost, and Latency of Inference Serving Systems
pjamshidi
0
130
Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems
pjamshidi
0
140
Learning from Valerie Issarny: Insights Gained from Program Co-Chairing SEAMS’23
pjamshidi
0
270
Artificial Intelligence and Systems Laboratory (AISys): A Research Overview
pjamshidi
0
580
Experiential Learning by Building Real-World AI Systems
pjamshidi
0
210
Understanding and Explaining the Root Causes of Performance Faults with Causal AI: A Path towards Building Dependable Computer Systems
pjamshidi
0
160
On Debugging the Performance of Configurable Software Systems: Developer Needs and Tailored Tool Support
pjamshidi
0
250
Unicorn: Reasoning about Configurable System Performance through the Lens of Causality
pjamshidi
0
440
Causal AI for Systems
pjamshidi
0
300
Other Decks in Science
See All in Science
Introduction to Image Processing: 2.Frequ
hachama
0
500
Transformers are Universal in Context Learners
gpeyre
0
770
ガウス過程回帰とベイズ最適化
nearme_tech
PRO
1
240
創薬における機械学習技術について
kanojikajino
16
5.1k
ACL読み会2024@名大 REANO: Optimising Retrieval-Augmented Reader Models through Knowledge Graph Generation
takuma_matsubara
0
180
最適化超入門
tkm2261
15
3.8k
メール送信サーバの集約における透過型SMTP プロキシの定量評価 / Quantitative Evaluation of Transparent SMTP Proxy in Email Sending Server Aggregation
linyows
0
770
All-in-One Bioinformatics Platform Realized with Snowflake ~ From In Silico Drug Discovery, Disease Variant Analysis, to Single-Cell RNA-seq
ktatsuya
0
330
白金鉱業Meetup Vol.16_数理最適化案件のはじめかた・すすめかた
brainpadpr
3
1.5k
06_浅井雄一郎_株式会社浅井農園代表取締役社長_紹介資料.pdf
sip3ristex
0
260
MoveItを使った産業用ロボット向け動作作成方法の紹介 / Introduction to creating motion for industrial robots using MoveIt
ry0_ka
0
350
第61回コンピュータビジョン勉強会「BioCLIP: A Vision Foundation Model for the Tree of Life」
x_ttyszk
1
1.7k
Featured
See All Featured
Side Projects
sachag
452
42k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
11
610
The Art of Programming - Codeland 2020
erikaheidi
53
13k
Measuring & Analyzing Core Web Vitals
bluesmoon
6
320
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.5k
VelocityConf: Rendering Performance Case Studies
addyosmani
328
24k
Facilitating Awesome Meetings
lara
53
6.3k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
4
490
Mobile First: as difficult as doing things right
swwweet
223
9.5k
Into the Great Unknown - MozCon
thekraken
35
1.7k
Practical Orchestrator
shlominoach
186
10k
Transcript
FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization of Deep Neural Networks
Shahriar Iqbal, Jianhai Su, Lars Kotthoff, Pooyan Jamshidi
[email protected]
AAAI, 24 February 2024 1
One Size Does Not Fit All 1 1.5 2 2.5
3 3.5 ·104 15 20 25 30 35 40 Energy Consumption (mJ) Prediction Error (%) Xception ← Energy consumption varies 4 × → ← Prediction Error varies 3 × → 2
Heterogeneous Parameters Num of Filters, Filter Size, Learning Rate, Num
of Epochs DN N Design Compiler Hardware Deployment Num of Active CPUs, CPU/ GPU/ EMC Frequency Cloud, IoT, Edge Num of Threads, GPU Threads, Memory Growth 3
Cost-Unaware Methods Waste Resources Coupled Unaware Pareto Optimal Prediction Error
(%) Log Wall Clock Time Energy Consumption (mJ) 3000 6000 9000 12000 15 25 35 45 3.65 3.50 3.35 Decoupled Aware Pareto Optimal Prediction Error (%) Log Wall Clock Time Energy Consumption (mJ) 3000 6000 9000 12000 15 25 35 45 3.65 3.50 3.35 4
Proposed Method ▷ weight expected benefit of evaluation by cost
▷ choose which objective(s) to evaluate ▷ more efficient use of resources – lower cost, more evaluations 5
Results – Computer Vision 0 50 100 150 200 Cumulative
Log WallClock Time 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 Hypervolume Error Xception PAL PESMO ParEGO SMSEGO CA-MOBO PESMO-DEC FLEXIBO-GPLC 10000 15000 20000 25000 Energy Consumption (mJ) 15 20 25 30 35 40 Prediction Error (%) Xception PAL PESMO ParEGO SMSEGO CA-MOBO PESMO-DEC FLEXIBO-GPLC 6
Results – NLP 0 50 100 150 200 Cumulative Log
WallClock Time 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 Hypervolume Error BERT-SQuAD PAL PESMO ParEGO SMSEGO CA-MOBO PESMO-DEC FLEXIBO-GPLC 20000 30000 40000 50000 60000 70000 80000 90000 Energy Consumption (mJ) 20 25 30 35 Prediction Error (%) BERT-SQuAD PAL PESMO ParEGO SMSEGO CA-MOBO PESMO-DEC FLEXIBO-GPLC 7
Results – Speech Recognition 0 50 100 150 200 250
300 Cumulative Log WallClock Time 0.25 0.30 0.35 0.40 0.45 0.50 0.55 Hypervolume Error DeepSpeech PAL PESMO ParEGO SMSEGO CA-MOBO PESMO-DEC FLEXIBO-GPLC 20000 30000 40000 50000 60000 Energy Consumption (mJ) 17.5 20.0 22.5 25.0 27.5 30.0 32.5 35.0 Prediction Error (%) DeepSpeech PAL PESMO ParEGO SMSEGO CA-MOBO PESMO-DEC FLEXIBO-GPLC 8
Results – Evaluations 0 20 40 60 80 100 120
140 160 180 200 PAL 0 20 40 60 80 100 120 140 160 180 200 PESMO-DEC 2 4 6 8 0 20 40 60 80 100 120 140 160 180 200 Iteration CA-MOBO 0 20 40 60 80 100 120 140 160 180 200 Iteration FlexiBO 2 4 6 8 9
FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization of Deep Neural Networks
▷ cost-aware acquisition function decreases cost and improves results ▷ code available at https://github.com/softsys4ai/FlexiBO 0 50 100 150 200 250 300 Cumulative Log WallClock Time 0.25 0.30 0.35 0.40 0.45 0.50 0.55 Hypervolume Error DeepSpeech PAL PESMO ParEGO SMSEGO CA-MOBO PESMO-DEC FLEXIBO-GPLC 20000 30000 40000 50000 60000 Energy Consumption (mJ) 17.5 20.0 22.5 25.0 27.5 30.0 32.5 35.0 Prediction Error (%) DeepSpeech PAL PESMO ParEGO SMSEGO CA-MOBO PESMO-DEC FLEXIBO-GPLC 10