Don't Fear the Threads: Simplify Your Life with JRuby

Slides form my talk at RubyNation 2012

1. don’t fear the threads simplify your life with JRuby david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com 1
2. { :me => { :tech_lead => “LivingSocial” } } 2
3. First Assignment 3
4. Write a 4
5. Command Line App Write a 4
7. Command Line App Write a 6
8. Command Line App Write a To charge credit cards faster 6
9. NO NEW HARDWARE 6
11. THREADS! 7
12. Are we maximizing our resources? 8
13. Are we maximizing our resources? 9
14. What problem are we solving? 10
15. Do some I/O Compute Stuff 11
16. Do some I/O 12
17. Do some I/O Can I do I/O? Read/Write Block YES NO 13
18. Do some I/O Block 14
19. Block 15
20. Let someone else work Block 15
21. Maximize Resources Block 16
23. Will our code be 17
24. Will our code be easy to write? 17
25. easy to understand? Will our code be easy to write? 17
26. easy to understand? easy to test? Will our code be easy to write? 17
27. Are we maximizing our resources? Are we maximizing our resources? 18
28. Run lots of processes 19
30. fork { # your code } 20
31. easy to understand 21
32. doesn’t maximize resources 22
33. Parent 23
34. Parent Parent’s Memory 23
35. Parent Parent’s Memory Child fork { Parent’s Memory (copy of) 23
36. Event-based I/O 24
38. require 'em-http' include EM::HttpRequest EM.run { # some of your code that does IO some_callback { |results_of_io| # the rest of your code that # uses the results } } 25
39. require 'em-http' include EM::HttpRequest EM.run { # some of your code that does IO some_callback { |results_of_io| # the rest of your code that # uses the results } } 25
40. Need to do I/O again? 26
42. require 'em-http' include EM::HttpRequest EM.run { # some of your code that does SOME of your I/O some_callback { |results_of_io| # use the results # now some more code that does some MORE I/O some_other_callback { |results_of_more_io| # use the results of THIS I/O } } } 27
44. require 'em-http' include EM::HttpRequest EM.run { # some of your code that does SOME of your I/O some_callback { |results_of_io| # use the results # now some more code that does some MORE I/O some_other_callback { |results_of_more_io| # use the results of THIS I/O # SERIOUSLY, more I/O?!??!@ How much do you need? yet_more_callbacks { |moar_resultz| # How many levels deep are we now?! } } } } 28
45. require 'em-http' include EM::HttpRequest EM.run { # some of your code that does SOME of your I/O some_callback { |results_of_io| # use the results # now some more code that does some MORE I/O some_other_callback { |results_of_more_io| # use the results of THIS I/O # SERIOUSLY, more I/O?!??!@ How much do you need? yet_more_callbacks { |moar_resultz| # How many levels deep are we now?! } } } } 28
46. NOT easy to understand 29
47. kinda maximizes resources 30
48. Entire call chain must be evented 31
49. Not true parallelism 32
50. Not true parallelism unless you run lots of processes 32
51. What’s parallelism? 33
52. Concurrency 34
53. Parallelism 35
54. Parallelism Code running at the same time as other code 35
55. Parallelism Impossible with only one CPU Code running at the same time as other code 35
56. Event-based I/O 36
57. NOT easy to understand kinda maximizes resources 37
58. Threads 38
60. Thread.new { # your code } 39
61. fork { # your code } 40
62. Thread.new { # your code } 41
63. Need to do I/O? 42
64. Need to do I/O? Not a problem 42
65. Need to do I/O THREE TIMES? 43
66. Need to do I/O THREE TIMES? Not a problem 43
67. What if I need to calculate π to the 2,345,123rd decimal place? 44
68. NOT A PROBLEM 44
69. Threads achieve true parallelism 45
70. easy to understand 46
71. maximizes resources 47
72. Threads don’t work in Ruby 48
73. Threads don’t work in Ruby…right? 48
74. MRI (aka C Ruby) 49
75. MRI (aka C Ruby) Thread is an OS thread (except in 1.8) 49
76. MRI (aka C Ruby) Thread is an OS thread GIL :( (except in 1.8) 49
77. MRI (aka C Ruby) Thread is an OS thread GIL :( I/O will cause a context switch (except in 1.8) 49
78. Rubinius 50
79. Rubinius Thread is an OS thread 50
80. Rubinius Thread is an OS thread No GIL! 50
81. Rubinius Thread is an OS thread No GIL! True parallelism 50
82. JRuby 51
83. JRuby Thread is a JVM Thread (which is an OS thread) 51
84. JRuby Thread is a JVM Thread (which is an OS thread) Context switch for variety of reasons 51
85. JRuby Thread is a JVM Thread (which is an OS thread) Context switch for variety of reasons True parallelism 51
86. JRuby BATTLE TESTED 51
87. So, you’ve decided to use Threads… 52
88. You need to know four things 53
90. Start & Manage 54
91. Start & Manage (Dealing with) Shared State 54
92. Start & Manage (Dealing with) Shared State Using Third Party Libraries 54
93. Start & Manage (Dealing with) Shared State Using Third Party Libraries Context Switching 54
94. Start & Manage Threads 55
95. Chaos 56
96. Thread.new { # your code } Thread.new { # moar code } # etc 57
97. Hand-Roll 58
98. threads = [] threads << Thread.new { # your code } threads << Thread.new { # moar code } # etc threads.each(&:join) # All threads have completed exit 0 59
99. java.util.concurrent 60
100. import java.util.concurrent # JRuby only service = Executors.new_fixed_thread_pool(100) service.execute { # your code } service.execute { # some other code } 61
103. <T> 63
104. ExecutorService 64
105. ExecutorService 65
106. ExecutorService execute() 65
107. ExecutorService execute() shutdown() 65
108. ExecutorService execute() shutdown() await_termination() 65
109. ExecutorService execute() shutdown() await_termination() shutdown_now() 65
110. service = Executors.new_fixed_thread_pool(10) tcp_erver = TCPServer.new("127.0.0.1",8080) loop do { s = tcp_server.accept service.execute { s.puts calculate_pi() s.close } } 66
111. service = Executors.new_fixed_thread_pool(10) tcp_server = TCPServer.new("127.0.0.1",8080) Signal.trap('SIGINT') { service.shutdown } loop do { s = tcp_server.accept service.execute { s.puts calculate_pi() s.close } break if service.is_shutdown } 67
112. service = Executors.new_fixed_thread_pool(10) tcp_server = TCPServer.new("127.0.0.1",8080) Signal.trap('SIGINT') { service.shutdown } loop do { s = tcp_server.accept service.execute { s.puts calculate_pi() s.close } break if service.is_shutdown } service.await_termination(10,TimeUnit.SECONDS) service.shutdown_now 68
113. So much more 69
114. ScheduledExecutorService So much more 69
115. ScheduledExecutorService So much more ThreadFactory 69
116. Read the javadocs ScheduledExecutorService So much more ThreadFactory 69
117. Shared State 70
118. results = [] Thread.new { results << some_result() } Thread.new { results << other_result() } 71
119. mutex = Mutex.new results = [] Thread.new { mutex.synchronize { results << some_result() } } Thread.new { mutex.synchronize { results << other_result() } } 72
120. Avoid explicit locking/sync 73
121. results = ConcurrentLinkedQueue.new Thread.new { results.offer(some_result()) } Thread.new { results.offer(some_result()) } 74
122. import java.util.concurrent.atomic flag = AtomicBoolean.new(true) Thread.new { if flag.get { flag.set(false) } } Thread.new { flag.set(true) } 75
123. So much more 76
124. ConcurrentHashMap So much more 76
125. ConcurrentHashMap So much more AtomicReference 76
126. Read the javadocs ConcurrentHashMap So much more AtomicReference 76
127. Third Party Libraries 77
128. Are they thread safe? 78
129. Probably 79
130. Probably …but to be sure 79
131. variables 80
132. variables Global 80
133. variables Global Class 80
134. Most are fine 81
135. Context Switching 82
136. Amdahl’s Law 83
137. Performance Gains Increase in # of Threads Speedup 84
138. Performance Gains with Context Switching Increase in # of Threads Speedup Cost of Context Switching Takes Over 85
139. Thread Pools 86
140. Worker Worker Worker Worker Thread Thread Thread Thread Pool Worker Worker Worker Worker 87
141. Worker Worker Worker Worker Thread Thread Thread Thread Pool Worker Worker Worker Worker 87
142. Worker Worker Worker Worker Thread Thread Thread Thread Pool Worker Worker Worker 87
143. Thread.new { # your code } 88
144. service.execute { # your code } 89
145. Not usually a concern 90
146. Always use Threads, right? 91
147. Thread UNsafe Libraries 92
148. Evented 93
149. Simple, I/O bound task 94
150. Evented 95
151. Cannot use JRuby 96
152. Evented 97
153. Otherwise, Threads will simplify your code and maximize your resources 98
154. Semantic layers on top of evented I/O 99
156. You’re using Threads 100
157. You’re using Threads (in degenerate form) 100
158. Exercises 101
159. Echo Server •Listen on a port •Respond to each request in a new Thread •Extra Credit: Record stats on requests in a shared data structure 102
160. Connection Pool •Allow N clients to access X shared instances of, say, Redis (where N > X) •Clients “check out” a connection and get exclusive access •Clients “check in” when done •Instances get re-used 103
161. THANKS! david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com 104