Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
640
Introduction to Rake
drbrain
0
320
Lessons in Mentorship
drbrain
1
210
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
110
Open Source Maintenance — RailsClub Moscow
drbrain
1
160
drbdump
drbrain
2
490
Other Decks in Programming
See All in Programming
意外と簡単!?フロントエンドでパスキー認証を実現する WebAuthn
teamlab
PRO
2
770
Tool Catalog Agent for Bedrock AgentCore Gateway
licux
7
2.5k
2025 年のコーディングエージェントの現在地とエンジニアの仕事の変化について
azukiazusa1
24
12k
CJK and Unicode From a PHP Committer
youkidearitai
PRO
0
110
AI Agents: How Do They Work and How to Build Them @ Shift 2025
slobodan
0
100
Design Foundational Data Engineering Observability
sucitw
3
200
個人軟體時代
ethanhuang13
0
330
アルテニア コンサル/ITエンジニア向け 採用ピッチ資料
altenir
0
110
testingを眺める
matumoto
1
140
請來的 AI Agent 同事們在寫程式時,怎麼用 pytest 去除各種幻想與盲點
keitheis
0
130
AWS発のAIエディタKiroを使ってみた
iriikeita
1
190
速いWebフレームワークを作る
yusukebe
5
1.7k
Featured
See All Featured
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Writing Fast Ruby
sferik
628
62k
RailsConf 2023
tenderlove
30
1.2k
Embracing the Ebb and Flow
colly
87
4.8k
Raft: Consensus for Rubyists
vanstee
140
7.1k
How GitHub (no longer) Works
holman
315
140k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
3k
Large-scale JavaScript Application Architecture
addyosmani
513
110k
Imperfection Machines: The Place of Print at Facebook
scottboms
268
13k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Navigating Team Friction
lara
189
15k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)