Enumerator - Enumerable's Ugly Cousin

Why don't more Rubyists use Enumerator? Learn more about this beautiful Ruby concept that could use a little more love.

Transcript

1. Enumerator Enumerable’s Ugly Cousin
2. 2 @rossta NYC
3. Pretend everything is normal NYC
4. Choose one of two speeds: Move fast or… NYC
5. GET OUT OF THE WAY NYC
6. Question yourself at every turn… NYC
7. NYC
9. What we say about Enumerable
10. Powerful, simple, and elegant. How modules should be made! Learn it. Use it.
11. Why I fell in love with Ruby
12. What we say about Enumerator
13. I don’t get it I would never write code this way That’s so ugly
14. This seems like a big hack Why would I ever use this?
15. OMFG
17. PRACTICAL EXPRESSIVE CLEAN ELEGANT MINASWAN READABLE SIMPLE SIMPLE HAPPINESS BEAUTIFUL CONCISE
18. Conventions are great because Productivity!
19. Conventions can limit our growth But…
20. for loops are useful! C++ MATLAB Perl Algol BASIC PostScript C Pascal Ada Bash Haskell Python Lua Java JavaScript PHP ActionScript Go
21. for n in [1, 2, 3] # ... end
22. “Real” Rubyists use each, right? [1, 2, 3].each { |x| # ... }
23. – Zed Shaw “When you teach people social norms as if they are universal truths you are actually indoctrinating them not educating them. .”
24. “Code Morality” The “right” way and everything else
30. Social Norms + Conventions !== Universal truth
31. Yes, follow conventions! but…
32. Also explore unconventional ideas
33. Unconventional Ugly
34. What is Enumerator?
35. Enumerator – Ruby Docs “A class which allows both internal and external iteration.”
36. enum = [1, 2, 3].to_enum # => #<Enumerator: [1, 2, 3]:each> enum.next # => 1 enum.next # => 2 enum.next # => 3 enum.next # => StopIteration enum.each { |x| # ... }
37. Let’s explore
38. Java What I learned from
39. Iterator
40. ArrayList list = list.add( list.add( list.add( ArrayList list = new ArrayList(); list.add(1); list.add(2); list.add(3); Iterator it = list.iterator(); while(it.hasNext()) { Object element = it.next(); System.out.println(element); };
41. ArrayList list = list.add( list.add( list.add( Iterator it = list.iterator(); while Object element = it.next(); System.out.println(element); }; it.next();
42. Iterator an object that can suspend iteration and pass control back to its caller
43. Iterator Like yield turned inside out
44. def each yield 1 # to block yield 2 # to block yield 3 # to block end enum = [1, 2, 3].to_enum enum.next # yield 1 enum.next # yield 2 enum.next # yield 3
45. Use case: Replace nested loops
46. <table> <% projects.each do |project| %> <tr style="background: <%= colors.next %>"> <td><%= project.name %></td> </tr> <% end %> </table> <% colors = %w[aliceblue ghostwhite].to_enum(:cycle) %>
47. <% colors = <table> <% <% </table> colors.next
48. What I learned from JavaScript & Python
49. Generator
50. Iterators Generators
51. Generators #next
52. Generators produce data on the fly
53. async infinite sequences fetching data list comprehensions concurrency lazy evaluation coroutines
54. JavaScript Generators in
56. –AirBnB JavaScript Style Guide “Don't use generators.”
61. Python Generators in
62. Awesome Simple, expressive. Love them. List comprehensions… insanely readable and easy to maintain
63. gen = python_generator() gen.next() # => 1 gen.next() # => 2 gen.next() # => 3 def python_generator(): yield 1 yield 2 yield 3
64. def fibonacci(n): result = []
65. def fibonacci(n): result = [] a = b = 1 for i in xrange(n): result.append(a) a, b = b, a + b return result
66. def fibonacci(n): result = [] a = b = 1 for i in xrange(n): result.append(a) a, b = b, a + b return result for a in fibonacci(20): print a
67. def fibonacci(n): a = b = 1 for i in xrange(n): a, b = b, a + b for a in fibonacci(20): print a
68. def fibonacci(n): a = b = 1 for i in xrange(n): yield a a, b = b, a + b for a in fibonacci(20): print a
69. gen = fibonacci(20) gen.next() gen.next() gen.next() def a = b = a, b = b, a + b
70. Generator a function that can suspend its execution and pass control back to its caller
71. Generator Provides ability to treat algorithms as collections
72. Can we write generators in Ruby?* *yes, we can! Q:
73. def fibonacci(n): a = b = 1 for i in xrange(n): yield a a, b = b, a + b for a in fibonacci(20): print a
74. def fibonacci(n) a = b = 1 n.times do yield a a, b = b, a + b end end fibonacci(20) { |a| puts a } Not enumerable!
75. Enumeratorize it!
76. def fibonacci(n) a = b = 1 n.times do yield a a, b = b, a + b end end def a = b = n.times a, b = b, a + b end return to_enum(__method__, n) unless block_given?
77. return to_enum(__method__) unless block_given? This seems like a big hack Too magical That’s ugly
78. def fibonacci(n) a = b = 1 n.times do yield a a, b = b, a + b end end def a = b = n.times a, b = b, a + b end fibonacci(7).each { |a| puts a } fibonacci(25).map { |a| a * 3 }.select(&:odd?) fibonacci(100).find { |a| a > 50 } return to_enum(__method__, n) unless block_given? ] [ Enumerable!
79. Enumerable + Generator == Enumerator
80. to_enum
81. to_enum is everywhere
82. [1, 2, 3].to_enum(:each) # => #<Enumerator: [1, 2, 3]:each> [1, 2, 3].to_enum(:map) # => #<Enumerator: [1, 2, 3]:map> [1, 2, 3].to_enum(:select) # => #<Enumerator: [1, 2, 3]:select> [1, 2, 3].to_enum(:group_by) # => #<Enumerator: [1, 2, 3]:group_by> [1, 2, 3].to_enum(:cycle) # => #<Enumerator: [1, 2, 3]:cycle>
83. [1, 2, 3].each # => #<Enumerator: [1, 2, 3]:each> [1, 2, 3].map # => #<Enumerator: [1, 2, 3]:map> [1, 2, 3].select # => #<Enumerator: [1, 2, 3]:select> [1, 2, 3].group_by # => #<Enumerator: [1, 2, 3]:group_by> [1, 2, 3].cycle # => #<Enumerator: [1, 2, 3]:cycle>
84. Fixnum 4.times 1.upto(10) 10.upto(1) Range (1..10).each (1..10).step Struct struct.each struct.each_pair String "123".each_byte "123".each_char "123".each_codepoint "123".each_line "123".gsub File File.foreach file.each file.each_line file.each_byte file.each_codepoint ObjectSpace ObjectSpace.each_object loop
85. Object#to_enum Not magic!
86. We can re-implement to_enum in Ruby
87. obj_to_enum(int argc, VALUE *argv, VALUE obj) { VALUE enumerator, meth = sym_each; if (argc > 0) { --argc; meth = *argv++; } enumerator = rb_enumeratorize_with_size(obj, meth, argc, argv, 0); if (rb_block_given_p()) { enumerator_ptr(enumerator)->size = rb_block_proc(); } return enumerator; } static VALUE enumerator.c
88. obj_to_enum( { VALUE enumerator, meth = sym_each; --argc; meth = *argv++; } enumerator = rb_enumeratorize_with_size(obj, meth, argc, argv, enumerator_ptr(enumerator)->size = rb_block_proc(); } return enumerator; } rb_enumeratorize_with_size(obj, meth, static VALUE enumerator.c
89. VALUE rb_enumeratorize_with_size(VALUE obj, VALUE meth, int *size_fn) { dispatching to either return lazy_to_enum_i(obj, meth, argc, argv, size_fn); return enumerator_init( enumerator_allocate(rb_cEnumerator), } return enumerator_init( enumerator_allocate(rb_cEnumerator), obj, meth, argc, argv, size_fn, Qnil); rb_enumeratorize_with_size
90. def to_enum(method, *args) Enumerator.new(self, method, *args) end
91. Enumerator Collection Enumerable method
93. static next_i { VALUE nil VALUE result; result next_ii e "iteration reached an end" return } next_i(VALUE curr, VALUE obj) return rb_fiber_yield(1, &nil); enumerator.c Fiber!
94. We can re-implement Enumerator with Fiber
95. Fiber.new fiber = Fiber.new do Fiber.yield 1 Fiber.yield 2 Fiber.yield 3 end fiber.resume # => 1 fiber.resume # => 2 fiber.resume # => 3
96. Look familiar? Q:
97. gen = python_generator() gen.next() # => 1 gen.next() # => 2 gen.next() # => 3 def python_generator(): yield 1 yield 2 yield 3
98. Fiber.new fiber = Fiber.new do Fiber.yield 1 Fiber.yield 2 Fiber.yield 3 end fiber.resume # => 1 fiber.resume # => 2 fiber.resume # => 3
99. a block that can suspend its execution and pass control back to its caller Fiber
100. Fiber =~ Generator
101. def to_enum(method, *args) Enumerator.new(self, method, *args) end
102. class CustomEnumerator def initialize(collection, meth, *args) @collection = collection @fiber = Fiber.new do @collection.send(meth, *args) do |n| Fiber.yield(n) end raise StopIteration end end end class CustomEnumerator def initialize(collection, meth, *args) @collection = collection class CustomEnumerator
103. @fiber = Fiber.new do @collection.send(meth, *args) do |n| Fiber.yield(n) end raise StopIteration end class def end
104. Enumerator Fiber Collection Enumerable method
105. class CustomEnumerator def next @fiber.resume end end
106. class CustomEnumerator include Enumerable def next @fiber.resume end def each loop { yield self.next } end end # => StopIteration
107. Enumerable + Fiber == Enumerator
108. Should we “enumeratorize” by convention? Q:
109. class PaginatedApiClient def posts(page=0) return to_enum(:posts, page) unless bg? # fetch page # yield each post # call posts(page+1) end end
110. class Document include Enumerable def each # yield each end end paragraph word line class Document
111. class Document def each_line return to_enum(:each_line) unless block_given? # yield each line end def each_word return to_enum(:each_word) unless block_given? # yield each word end def each_para return to_enum(:each_para) unless block_given? # yield each paragraph end end class Document def each_line return to_enum(:each_line) unless block_given? # yield each line end class Document def each_line return to_enum(:each_line) unless block_given? # yield each line end def each_word return to_enum(:each_word) unless block_given? # yield each word end class Document
112. class BinaryTree def breadth_first return to_enum(__method__) unless block_given? # yield values in “breadth first” order end def pre_order # etc… end def post_order end def in_order end end
113. tree = tree.breadth_first. with_index. partition { |n, i| i.odd? }. flat_map(&:join) # => “b1d3f5”, “a0c2e4”
114. Ugly but effective return to_enum(__method__) unless block_given? tree.breadth_first. with_index. partition { #… } flat_map { #… }
115. What I learned from Clojure & Elixir
116. Infinite Sequence
117. (take 2 (filter odd? [0 1 2 3 4 5])) [0, 1, 2, 3, 4, 5] |> Enum.filter(&Integer.is_odd/1) |> Enum.take(2)
118. (take 2 (filter odd? (iterate inc 0))) Stream.iterate(0, &(&1 + 1)) |> Stream.filter(&Integer.is_odd/1) |> Enum.take(2)
119. We don’t have infinite sequences in Ruby… or do we?
120. def fibonacci Enumerator.new do |y| a = b = 1 loop do y.yield a a, b = b, a + b end end end like Fiber.yield!
121. def a = b = y.yield a a, b = b, a + b end fibonacci.lazy. select(&:odd?). take(3)
122. lazy augments how data is processed in an enumerator chain
123. Eager Pipeline [] map filter sum
124. Lazy Pipeline [] map filter sum
125. Failed Project What I learned from a
126. Recurrence
128. What’s the estimate? Oh, about 4 days
129. 4 days weeks
130. Modeling Recurrence
131. { every: :month, on: { friday: 13 } }
132. Present Day: Redemption
133. Wish list enumerable lazily generate events infinite recurrence queryable
134. i.e., an algorithm for an enumerable infinite recurring events
135. Enumeratorize it!
136. $ gem install montrose
137. Recurrence Enumerator find next event yield stop or continue
138. # Expressive Montrose.weekly(on: :monday, at: "9 am") # => #<Montrose::Recurrence...> # Flexible Montrose.hourly.interval(3) Montrose.every(3.hours) Montrose.r(every: 3.hours) # Chainable Montrose.monthly. starting(1.year.from_now). on(friday: 13). repeat(5)
139. # Enumerable r = Montrose.monthly r.until(1.year.from_now).each do |event| # … end r.lazy.chunk(&:month).take(8).to_a # => [ [4, 2016-04-06 12:00:00 -0500], [5, 2016-05—06 12:00:00 -0500], [6, 2016-06—06 12:00:00 -0500], ...]
140. Enumerator is beautiful
141. fibers infinite sequences generators iterators lazy evaluation Enumerator Enumerable
142. This wasn’t just a talk about Enumerator
143. –Me “I wasn’t capable of writing Montrose a few years go. I needed to get out of my comfort zone.”
144. MY RUBY GROWTH CYCLE Uninformed Optimism Informed Pessimism Informed Optimism
146. “WTF?” Instead of
147. “What can I learn from this?” Try
148. Sometimes, ugly code can teach us something
149. Go forth and be curious
150. @rossta rossta.net/talks Ross Kaffenberger