Memory Leaks, Tweaks, and Techniques

Transcript

1. Memory Leaks 2015 @schneems
2. Leaks are bad
7. Obviously memory leaks are bad
8. Why?
9. First…
10. They Call me @Schneems
13. “Favor composition over inheritance
14. Golang rulz Elixr 4 lyfe
15. Rails Commit Bit
20. I like working with wood
22. Plan for seasonal wood movement “
23. Why is memory important?
24. RAM is a limited resource
25. When you run out, the OS starts using the HD
26. Hard Drive
27. RAM
28. swap
29. When you run out of room…
32. Performance Profiling
33. Start with a known
34. Repro the performance problem
35. #include <stdlib.h> void f(void) { int* x = malloc(10 *sizeof(int)); x[10] = 0; } int main(void) { f(); return 0; }
36. #include <stdlib.h> void f(void) { int* x = malloc(10 *sizeof(int)); x[10] = 0; } int main(void) { f(); return 0; }
37. Find the right tools
38. $ valgrind --tool=memcheck ./memoryleak ==31747== Memcheck, a memory error detector ==31747== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==31747== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==31747== Command: ./memoryleak ==31747== ==31747== Invalid write of size 4 ==31747== at 0x100000F4C: f (memoryleak.c:6) ==31747== by 0x100000F73: main (memoryleak.c:10) ==31747== Address 0x100ae34a8 is 0 bytes after a block of size 40 alloc'd ==31747== at 0x100008EBB: malloc (in /usr/local/Cellar/valgrind/3.11.0/lib/ valgrind/vgpreload_memcheck-amd64-darwin.so) ==31747== by 0x100000F43: f (memoryleak.c:5) ==31747== by 0x100000F73: main (memoryleak.c:10) ==31747== ==31747== ==31747== HEAP SUMMARY: ==31747== in use at exit: 35,304 bytes in 431 blocks ==31747== total heap usage: 503 allocs, 72 frees, 41,272 bytes allocated ==31747== ==31747== LEAK SUMMARY: ==31747== definitely lost: 40 bytes in 1 blocks ==31747== indirectly lost: 0 bytes in 0 blocks ==31747== possibly lost: 0 bytes in 0 blocks ==31747== still reachable: 0 bytes in 0 blocks ==31747== suppressed: 35,264 bytes in 430 blocks ==31747== Rerun with --leak-check=full to see details of leaked memory ==31747== ==31747== For counts of detected and suppressed errors, rerun with: -v ==31747== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
39. $ valgrind --tool=memcheck ./memoryleak ==31747== Memcheck, a memory error detector ==31747== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==31747== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==31747== Command: ./memoryleak ==31747== ==31747== Invalid write of size 4 ==31747== at 0x100000F4C: f (memoryleak.c:6) ==31747== by 0x100000F73: main (memoryleak.c:10) ==31747== Address 0x100ae34a8 is 0 bytes after a block of size 40 alloc'd ==31747== at 0x100008EBB: malloc (in /usr/local/Cellar/valgrind/3.11.0/lib/ valgrind/vgpreload_memcheck-amd64-darwin.so) ==31747== by 0x100000F43: f (memoryleak.c:5) ==31747== by 0x100000F73: main (memoryleak.c:10) ==31747== ==31747== ==31747== HEAP SUMMARY: ==31747== in use at exit: 35,304 bytes in 431 blocks ==31747== total heap usage: 503 allocs, 72 frees, 41,272 bytes allocated ==31747== ==31747== LEAK SUMMARY: ==31747== definitely lost: 40 bytes in 1 blocks ==31747== indirectly lost: 0 bytes in 0 blocks ==31747== possibly lost: 0 bytes in 0 blocks ==31747== still reachable: 0 bytes in 0 blocks ==31747== suppressed: 35,264 bytes in 430 blocks ==31747== Rerun with --leak-check=full to see details of leaked memory ==31747== ==31747== For counts of detected and suppressed errors, rerun with: -v ==31747== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
40. $ valgrind --tool=memcheck ./memoryleak ==31747== Memcheck, a memory error detector ==31747== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==31747== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==31747== Command: ./memoryleak ==31747== ==31747== Invalid write of size 4 ==31747== at 0x100000F4C: f (memoryleak.c:6) ==31747== by 0x100000F73: main (memoryleak.c:10) ==31747== Address 0x100ae34a8 is 0 bytes after a block of size 40 alloc'd ==31747== at 0x100008EBB: malloc (in /usr/local/Cellar/valgrind/3.11.0/lib/ valgrind/vgpreload_memcheck-amd64-darwin.so) ==31747== by 0x100000F43: f (memoryleak.c:5) ==31747== by 0x100000F73: main (memoryleak.c:10) ==31747== ==31747== ==31747== HEAP SUMMARY: ==31747== in use at exit: 35,304 bytes in 431 blocks ==31747== total heap usage: 503 allocs, 72 frees, 41,272 bytes allocated ==31747== ==31747== LEAK SUMMARY: ==31747== definitely lost: 40 bytes in 1 blocks ==31747== indirectly lost: 0 bytes in 0 blocks ==31747== possibly lost: 0 bytes in 0 blocks ==31747== still reachable: 0 bytes in 0 blocks ==31747== suppressed: 35,264 bytes in 430 blocks ==31747== Rerun with --leak-check=full to see details of leaked memory ==31747== ==31747== For counts of detected and suppressed errors, rerun with: -v ==31747== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
41. Apply the tools to your App
42. How does Ruby Memory?
43. Ruby is a “managed” Language
44. Don’t manually allocate, use Garbage Collector instead
45. 2 causes for increased memory use
46. Retained Objects 1)
47. RETAINED = [] 1.upto(100_000_000).each do |i| RETAINED << “#{i}” end
48. 7.32 GB
49. WHY?
50. Memory
51. Memory Object slots
52. Memory “1” “2” “3”“4” “5” “6” “7” RETAINED = [] 1.upto(100_000_000).each do |i| RETAINED << “#{i}” end
53. Out of slots! Allocate more memory
54. Memory “1” “2” “3”“4” “5” “6” “7” “10” “8” “9” “11” “12” “13” “14”
55. Allocated Objects 2)
56. 1.upto(100_000_000).each do |i| puts "#{i}" end
57. 21.8 MB
58. Memory “1” “2” “3”“4” “5” “6” “7”
59. Out of slots! Run GC!
60. Memory “2” “3”“4” “5” “6” “7” 1.upto(100_000_000).each do |i| puts "#{i}" end “1”
61. Memory “2” “3”“4” “5” “6” “7” 1.upto(100_000_000).each do |i| puts "#{i}" end
62. Memory “8” “3”“4” “5” “6” “7” 1.upto(100_000_000).each do |i| puts "#{i}" end “2”
63. GC will save us!
64. def make_objects array = [] 1.upto(100_000_000).each do |i| array << "#{i}" end return nil end make_objects GC.start(full_mark: true, immediate_sweep: true)
65. def make_objects array = [] 1.upto(100_000_000).each do |i| array << "#{i}" end return nil end make_objects GC.start(full_mark: true, immediate_sweep: true)
66. def make_objects array = [] 1.upto(100_000_000).each do |i| array << "#{i}" end return nil end make_objects GC.start(full_mark: true, immediate_sweep: true)
67. Result?
68. 7.32 GB
69. WAT
70. GC.start(full_mark: true, immediate_sweep: true) GC.disable # Code goes here before = GC.stat[:total_freed_objects] GC.enable GC.start(full_mark: true, immediate_sweep: true) after = GC.stat[:total_freed_objects] # => “Objects cleared by GC: 100_010_687”
71. Cannot open slots while reference exists
72. After program runs
73. Memory “1” “2” “3”“4” “5” “6” “7” “10” “8” “9” “11” “12” “13” “14”
74. Run GC
75. Memory “1” “2” “3”“4” “5” “6” “7” “10” “8” “9” “11” “12” “13” “14”
76. Ruby releases memory SLOWLY
77. Memory
78. Test Everything (even these slides)
79. How does this affect my Rails app?
80. Retained App
81. Retained Allocated App
82. Mem Usage App
83. Request Starts
84. Request Full .
85. Request Full Full ..
86. Request Full Full Full ...
87. Request Full Full Full Full ....
88. Request Full Full Full Full Full ......
89. Out of memory Full Full Full Full Full
90. Run GC, All objs needed 0_o Full Full Full Full Full
91. Allocate mem Full Full Full Full Full
92. Request Done Full Full Full Full Full ........
93. Eventually
94. GC Objs not used Full Full Full Full Full
95. GC Objs not used
96. Memory increases even without object retention
97. Object creation/ deletion takes RAM
98. Object creation/ deletion takes Time
99. Further Reading:
100. Generational GC
101. What is a memory leak?
102. What is a memory leak?
103. #include <stdlib.h> void f(void) { int* x = malloc(10 *sizeof(int)); x[10] = 0; } int main(void) { f(); return 0; }
104. We lost the reference
105. We can never free() that memory
106. RETAINED = [] 1.upto(100_000_000).each do |i| RETAINED << “#{i}” end
107. We retained the reference
108. We can never GC that memory
109. Apply the tools to your App
110. Do you have a memory leak?
113. Ruby memory should plateau
114. Do you have a memory leak?
115. No
116. Memory Bloat
117. Real life is messy
118. Easiest way to stop a memory bloat ???
120. Puma Worker Killer
121. Rolling Restarts
122. Memory Triggered Restarts
123. Doesn’t work with containers
124. Puma Worker Killer
125. Concurrency via processes
126. Reduce “worker” count to reduce RAM
127. Hardest way to stop memory bloat ???
128. Debugging and Introspection
129. Who has heard of derailed?
130. derailed_benchmarks
131. Memory at Boot
132. $ bundle exec derailed bundle:mem TOP: 54.1836 MiB mail: 18.9688 MiB mime/types: 17.4453 MiB mail/field: 0.4023 MiB mail/message: 0.3906 MiB action_view/view_paths: 0.4453 MiB action_view/base: 0.4336 MiB
133. Memory at Runtime
134. $ bundle exec derailed exec perf:objects # ... ## allocated memory by location --------------------------------------- 1935548 /Users/richardschneeman/.gem/ruby/2.2.3/gems/actionpack-4.2.3/lib/action_dispa 896100 /Users/richardschneeman/.gem/ruby/2.2.3/gems/pg-0.16.0/lib/pg/result.rb:10 741488 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerecord-4.2.3/lib/active_rec 689299 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activesupport-4.2.3/lib/active_su 660672 /Users/richardschneeman/.gem/ruby/2.2.3/gems/actionpack-4.2.3/lib/action_dispa 606384 /Users/richardschneeman/Documents/projects/codetriage/app/views/repos/\_repo.h 579384 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activesupport-4.2.3/lib/active\_s 532800 /Users/richardschneeman/.gem/ruby/2.2.3/gems/actionpack-4.2.3/lib/action_dispa 391392 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerecord-4.2.3/lib/active_rec 385920 /Users/richardschneeman/.gem/ruby/2.2.3/gems/temple-0.7.5/lib/temple/utils.rb:
135. Case Study
139. $ heroku pg:pull
140. $ PATH_TO_HIT=rails/rails bundle exec derailed exec perf:objects Booting: production Running 1 times Total allocated: 21495556 bytes (207735 objects) Total retained: 15041423 bytes (133220 objects) allocated memory by gem ----------------------------------- 19427636 activerecord-4.2.3 1471380 activesupport-4.2.3 214048 actionview-4.2.3 151066 actionpack-4.2.3 121983 codetriage/app 24308 will_paginate-3.0.7 23193 rack-1.6.4 21105 arel-6.0.3 13397 ruby-2.2.3/lib 5560 warden-1.2.3 5536 rack-timeout-0.3.2 3912 activemodel-4.2.3 3000 hashie-3.4.2 2154 devise-3.5.2 1573 omniauth-1.2.2 1536 i18n-0.7.0 1495 railties-4.2.3 787 temple-0.7.6
141. 19427636 bytes…
142. 19.4 MB
143. Per
144. Request
145. allocated memory by location ----------------------------------- 5131624 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec 4616480 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec 1336408 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec 1336088 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec 1330904 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec 1325648 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec 921807 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activesup 737376 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec 596288 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec 460720 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
146. ???
147. allocated objects by class ----------------------------------- 108939 String 35456 Hash 21605 Array 5890 Proc 5886 RubyVM::Env 5859 ActiveRecord::Attribute::FromDatabase 5759 ActiveRecord::AttributeSet 5759 ActiveRecord::LazyAttributeHash 5731 ActiveRecord::Associations::BelongsToAssociation 5731 Issue 217 ActiveSupport::SafeBuffer 173 MatchData 112 ActionView::OutputBuffer 59 Time 43 ActiveRecord::Relation 42 ActiveRecord::ConnectionAdapters::PostgreSQL::Name 27 ActiveRecord::AssociationRelation 27 User }
148. allocated objects by class ----------------------------------- 108939 String 35456 Hash 21605 Array 5890 Proc 5886 RubyVM::Env 5859 ActiveRecord::Attribute::FromDatabase 5759 ActiveRecord::AttributeSet 5759 ActiveRecord::LazyAttributeHash 5731 ActiveRecord::Associations::BelongsToAssociation 5731 Issue 217 ActiveSupport::SafeBuffer 173 MatchData 112 ActionView::OutputBuffer 59 Time 43 ActiveRecord::Relation 42 ActiveRecord::ConnectionAdapters::PostgreSQL::Name 27 ActiveRecord::AssociationRelation 27 User }
149. allocated objects by class ----------------------------------- 108939 String 35456 Hash 21605 Array 5890 Proc 5886 RubyVM::Env 5859 ActiveRecord::Attribute::FromDatabase 5759 ActiveRecord::AttributeSet 5759 ActiveRecord::LazyAttributeHash 5731 ActiveRecord::Associations::BelongsToAssociation 5731 Issue 217 ActiveSupport::SafeBuffer 173 MatchData 112 ActionView::OutputBuffer 59 Time 43 ActiveRecord::Relation 42 ActiveRecord::ConnectionAdapters::PostgreSQL::Name 27 ActiveRecord::AssociationRelation 27 User
150. 27 Users }
151. allocated objects by class ----------------------------------- 108939 String 35456 Hash 21605 Array 5890 Proc 5886 RubyVM::Env 5859 ActiveRecord::Attribute::FromDatabase 5759 ActiveRecord::AttributeSet 5759 ActiveRecord::LazyAttributeHash 5731 ActiveRecord::Associations::BelongsToAssociation 5731 Issue 217 ActiveSupport::SafeBuffer 173 MatchData 112 ActionView::OutputBuffer 59 Time 43 ActiveRecord::Relation 42 ActiveRecord::ConnectionAdapters::PostgreSQL::Name 27 ActiveRecord::AssociationRelation 27 User
152. NOT 5731 Issues }
154. Started GET "/rails/rails" for ::1 at 2015-11-04 14:54:55 -0600 ActiveRecord::SchemaMigration Load (19.2ms) SELECT "schema_migrations".* FROM "schema_migration Processing by ReposController#show as HTML Parameters: {"full_name"=>"rails/rails"} Repo Load (21.2ms) SELECT "repos".* FROM "repos" WHERE "repos"."full_name" = $1 LIMIT 1 [["fu source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=1006ms state=activ Issue Load (1091.1ms) SELECT "issues".* FROM "issues" WHERE "issues"."repo_id" IN (36) source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=2093ms state=activ source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=3101ms state=activ source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=4027ms state=activ (48.1ms) SELECT COUNT(*) FROM "users" INNER JOIN "repo_subscriptions" ON "users"."id" = "repo_ User Load (53.4ms) SELECT "users".* FROM "users" INNER JOIN "repo_subscriptions" ON "users"."i Rendered subscribers/_avatars.html.slim (138.9ms) CACHE (0.0ms) SELECT COUNT(*) FROM "users" INNER JOIN "repo_subscriptions" ON "users"."id" = "r (0.6ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "users" INNER JOIN "re 36]] Issue Load (41.8ms) SELECT "issues".* FROM "issues" WHERE "issues"."repo_id" = $1 AND "issues" (44.2ms) SELECT COUNT(*) FROM "issues" WHERE "issues"."repo_id" = $1 AND "issues"."state" = $2 Rendered repos/show.html.slim within layouts/application (496.5ms) source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=5005ms state=activ Rendered application/_head.html.slim (1360.4ms) Rendered application/_flashes.html.slim (15.9ms) Rendered application/_logo.html.slim (21.5ms) Rendered application/_nav.html.slim (73.0ms) Rendered application/_thoughtbot.html.slim (11.5ms) Rendered application/_footer.html.slim (36.4ms) Completed 200 OK in 5760ms (Views: 1889.1ms | ActiveRecord: 1323.2ms) source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=6004ms state=activ
155. Started GET "/rails/rails" for ::1 at 2015-11-04 14:54:55 -0600 ActiveRecord::SchemaMigration Load (19.2ms) SELECT "schema_migrations".* FROM "schema_migration Processing by ReposController#show as HTML Parameters: {"full_name"=>"rails/rails"} Repo Load (21.2ms) SELECT "repos".* FROM "repos" WHERE "repos"."full_name" = $1 LIMIT 1 [["fu source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=1006ms state=activ Issue Load (1091.1ms) SELECT "issues".* FROM "issues" WHERE "issues"."repo_id" IN (36) source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=2093ms state=activ source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=3101ms state=activ source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=4027ms state=activ (48.1ms) SELECT COUNT(*) FROM "users" INNER JOIN "repo_subscriptions" ON "users"."id" = "repo_ User Load (53.4ms) SELECT "users".* FROM "users" INNER JOIN "repo_subscriptions" ON "users"."i Rendered subscribers/_avatars.html.slim (138.9ms) CACHE (0.0ms) SELECT COUNT(*) FROM "users" INNER JOIN "repo_subscriptions" ON "users"."id" = "r (0.6ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "users" INNER JOIN "re 36]] Issue Load (41.8ms) SELECT "issues".* FROM "issues" WHERE "issues"."repo_id" = $1 AND "issues" (44.2ms) SELECT COUNT(*) FROM "issues" WHERE "issues"."repo_id" = $1 AND "issues"."state" = $2 Rendered repos/show.html.slim within layouts/application (496.5ms) source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=5005ms state=activ Rendered application/_head.html.slim (1360.4ms) Rendered application/_flashes.html.slim (15.9ms) Rendered application/_logo.html.slim (21.5ms) Rendered application/_nav.html.slim (73.0ms) Rendered application/_thoughtbot.html.slim (11.5ms) Rendered application/_footer.html.slim (36.4ms) Completed 200 OK in 5760ms (Views: 1889.1ms | ActiveRecord: 1323.2ms) source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=6004ms state=activ
156. Loading EVERY issue for rails/rails repo
158. def self.find_by_full_name(full_name) Repo.includes(:issues).find_by!(full_name: full_name) end
159. def self.find_by_full_name(full_name) Repo.includes(:issues).find_by!(full_name: full_name) end
160. Screenshot 2015-10-22 09.39.20
161. Not a memory leak
162. Objects were getting Collected via GC
163. Puma Threads
164. Idle
165. Now Total
166. Requests
167. rails/rail
168. Now Total
169. saltstack/salt
170. Now Total
171. idle
172. Now Total
173. rustlang/rust
174. golang/go
175. isacs/npm
176. Now Total
177. Idle
178. Now Total
179. Was that necessary?
181. /rails/rails Unused Eager Loading detected Repo => [:issues] Remove from your finder: :includes => [:issues]
182. No
183. What if there was another cause?
184. Proof that memory tooling provides results
185. My
 Secret
186. https://www.dropbox.com/s/vyd7teot0pg349q/Screenshot%202015-11-05%2011.34.04.png?dl=0
187. $ bundle gem derailed_benchmarks
188. mime-types boot
189. https://www.dropbox.com/s/wdabatbn0crbgw2/Screenshot%202015-11-05%2011.54.12.png?dl=0
190. Thanks @jeremyevans
191. gem “mime-types”, “~> 2.6.1”, require: “mime/types/columnar”
192. https://www.dropbox.com/s/56bw1mewpm7rc65/Screenshot%202015-11-05%2011.38.15.png?dl=0 Omniauth 28% faster
193. https://www.dropbox.com/s/0n72bg81s0rsfdh/Screenshot%202015-11-05%2011.45.58.png?dl=0 36% smaller require memory
194. gem “mail”, “~> 2.6.2”
195. https://www.dropbox.com/s/7j0m6zxl27yl7e7/Screenshot%202015-11-05%2011.47.07.png?dl=0 10% faster requests in Rails
196. 1+ years of building and using tools
197. I made results however
198. All these changes
200. Now I can reproduce the a basic memory problem at will
201. What did I learn from 1+ years of PRs?
202. Our community has almost zero knowledge of how our programs use memory
203. Most apps are slow due to app code, not framework
204. Your Apps
205. Monitor Production Memory
206. https://www.dropbox.com/s/tzxhepl8o8gizw0/Screenshot%202015-11-05%2014.46.06.png?dl=0
207. Benchmark!
208. $ gem install derailed
209. Puma Worker Killer
210. $ gem install bullet
211. The Future
212. I want a memory tool as easy to use as Ruby
213. Using Heap Dumps to take Production snapshots
214. Better heap analysis
215. Getting this to work with Heroku
216. Can we make this better?
217. allocated memory by location ----------------------------------- 5131624 active_record/result.rb:116 1336408 active_record/attribute_set/builder.rb:30 1336088 active_record/persistence.rb:69 1330904 active_record/core.rb:547 1325648 active_record/attribute_methods.rb:359 921807 active_support/callbacks.rb:81 737376 active_record/core.rb:114 596288 active_record/associations.rb:162 460720 active_record/attribute_set/builder.rb:16
218. Stack information in heap dumps ???
219. What do other languages do ???
220. https://www.dropbox.com/s/rzs0owrqq3rja96/Screenshot%202015-11-05%2014.32.52.png?dl=0
221. https://www.dropbox.com/s/k0j31exhh2ir9oj/Screenshot%202015-11-05%2015.12.49.png?dl=0
222. Can we leverage existing tooling for C?
223. Valgrind ?
224. GDB ?
225. Visual debuggers?
226. Further Reading
230. One last thing
232. PSA: Sprockets is now owned by Rails
233. Refactors on the Horizon
234. 21 commits makes me #4 contributor
235. Let’s save sprockets