Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
620
Introduction to Rake
drbrain
0
300
Lessons in Mentorship
drbrain
1
190
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
100
Open Source Maintenance — RailsClub Moscow
drbrain
1
150
drbdump
drbrain
2
470
Other Decks in Programming
See All in Programming
Interface vs Types ~型推論が過多推論~
hirokiomote
1
190
型安全なDrag and Dropの設計を考える
yudppp
5
620
がんばりすぎないコーディングルール運用術
tsukakei
1
110
tsconfigのオプションで変わる型世界
keisukeikeda
1
110
なぜHono×GraphQLを選んだのか?
junichi_fukushima
0
830
マテリアルって何者?RealityKitで扱うマテリアル入門
nao_randd
0
130
型付け力を強化するための Hoogle のすゝめ / Boosting Your Type Mastery with Hoogle
guvalif
1
210
JVM の仕組みを理解して PHP で実装してみよう
m3m0r7
PRO
1
230
【TSkaigi 2025】これは型破り?型安全? 真実はいつもひとつ!(じゃないかもしれない)TypeScript クイズ〜〜〜〜!!!!!
kimitashoichi
1
280
推論された型の移植性エラーTS2742に挑む
teamlab
PRO
0
120
#QiitaBash TDDでAIに設計イメージを伝える
ryosukedtomita
2
1k
データと事例で振り返るDevin導入の"リアル" / The Realities of Devin Reflected in Data and Case Studies
rkaga
3
3.3k
Featured
See All Featured
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.5k
Agile that works and the tools we love
rasmusluckow
329
21k
Into the Great Unknown - MozCon
thekraken
38
1.8k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Fireside Chat
paigeccino
37
3.5k
The Invisible Side of Design
smashingmag
299
50k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Code Review Best Practice
trishagee
68
18k
Writing Fast Ruby
sferik
628
61k
Docker and Python
trallard
44
3.4k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3k
Automating Front-end Workflow
addyosmani
1370
200k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)