Refactoring with Science

Using data and Science to refactor with greater confidence.

1. Refactoring GitHub with Science ! #BigRubyConf
2. ! Hello, my name is Wynn.
3. ! Folks call me @pengwynn.
4. ! http://wynn.fm
5. Changing code is EASY! !
6. Changing code with CONFIDENCE isn't. !
7. How do we gain CONFIDENCE? !
8. Write lots of TESTS. ! ❯ testrb test/integration/api/repos_test.rb Loading suite in dotcom mode ... Run options: # Running tests: Finished tests in 30.638101s, 4.7979 tests/s, 64.7886 assertions/s. 147 tests, 1985 assertions, 0 failures, 0 errors, 0 skips ruby -v: ruby 2.1.0p0-github (development) [x86_64-darwin13.0]
9. Use data to inform decisions. !
10. ! /graph me -1hr @api.queries.serialized >
11. The Graph Store ! !
12. The Graph Store ! ! There's a graph for that™
13. The Graph Store ! ! There's a graph for that™ …if not, make one.
14. ! Built on Graphite.
15. !
16. !
17. ! # counter GitHub.stats.increment("the-thing.the-action") ! # gauge GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading) ! # timing GitHub.stats.time("the-thing.the-operation") { the_operation }
18. ! # counter GitHub.stats.increment("the-thing.the-action") ! # gauge GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading) ! # timing GitHub.stats.time("the-thing.the-operation") { the_operation }
19. ! # counter GitHub.stats.increment("the-thing.the-action") ! # gauge GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading) ! # timing GitHub.stats.time("the-thing.the-operation") { the_operation }
20. Deploy with confidence. " !
21. How# has "'d over the years. !
22. 1989 10 PRINT "WYNN" 20 GOTO 10
24. 1999 !
25. 2004 !
26. 2009 ! # General configuration settings, required for all recipes! set :application, "project-name" set :domain, "project-domain" set :extra_domains, %w() # Add something like www.project-domain.com role :app, domain role :web, domain role :db, domain, :primary => true ! # set :user, "user" ! # Deployment Settings set :repository, "repository url" set :deploy_to, "/absolute/path/to/location/your/server" set :deploy_via, :checkout set :config_files, %w() ! # SSH Keys for caching (you must generate these first.) ssh_options[:keys] = %w(~/.ssh/mykey1 ~/.ssh/mykey2) ! # Change this to :thin if you want to use Thin instead. set :app_server, :mongrel ! # Change this to :merb if you want to use Merb instead. (experimental) set :app_framework, :rails ! # ============================================================= # Application Server Settings (Thin or Mongrel) # ============================================================= set :app_servers, 1 set :app_server_port, 7007 set :app_environment, 'production' set :app_server_address, '127.0.0.1' set :app_server_conf, "#{shared_path}/config/thin.yml" ! # ============================================================= # Nginx Settings
27. Today !
28. ! The GitHub Flow
29. ! Start with a Pull Request.
30. !
31. ! Create a branch.
32. ! Add commits.
33. ! Open a pull request.
34. ! Discuss and review.
35. ! Merge and deploy.
36. ! Or deploy and merge. $
37. ! %
38. !
39. ! The following deployments are based on actual events. Branch names and SHAs have been changed to protect the innocent.
40. ! hubot where can I deploy? > Environments for github ----------------------------------------------------- production: unlocked lab-one: locked 6 hours ago by jasonrudolph lab-two: unlocked sekret-lab: unlocked playground: unlocked
41. ! hubot deploy github/api-my-feature to production >
42. ! hubot deploy github/api-my-feature to production > project
43. ! hubot deploy github/api-my-feature to production > branch
44. ! hubot deploy github/api-my-feature to production > environment
45. ! hubot deploy github/api-my-feature to production > Merge master into api-my-feature for pengwynn - hubot github/github@deadbeef
46. ! Get up to date with master. ⏳
47. ! Wait on the CI server. '
48. !
49. ! hubot tv me >
50. ! hubot deploy github/api-my-feature to production > Build #8675309 (deadbeef) of github/api-my-feature was successful (203s)
51. ! Release the kraken! (
52. ! hubot deploy github/api-my-feature to production > pengwynn is deploying github/api-my-feature (beefdead..deadbeef) to production pengwynn's production deployment of github is NOW )))! (22s)
53. "The first transport is away!"
54. ## TODO - [x] Deploy " - [ ] Observe * !
55. !
56. !
57. !
58. !
59. ! Problem? ABORT. +
60. ! hubot deploy! github > pengwynn is deploying github/master to production
61. ! ✨Everything -? MERGE.✨
62. !
63. ! > pengwynn: it looks like you merged "api-my-feature" branch into master, so I've unlocked production.
64. ! > pengwynn is deploying github/master to production pengwynn is deploying github/master to lab-one pengwynn is deploying github/master to lab-two pengwynn is deploying github/master to sekret-lab
65. Prune with confidence. ✂️ !
66. ! “Are we even calling this anywhere?”
67. ! Find and remove vestigial code.
68. ! GitHub::Backscatter
69. ! GitHub::Backscatter not open source, but you can roll your own
70. ! *69 for Ruby methods
71. class Foo include GitHub::Backscatter ! def the_method(args) backscatter_measure # ... end end
72. class Foo include GitHub::Backscatter ! def the_method(args) backscatter_measure # ... end end
73. class Foo include GitHub::Backscatter ! def the_method(args) backscatter_measure # ... end end
75. The calls are coming from INSIDE the APP. 0
76. class Foo include GitHub::Backscatter ! def the_method(args) backscatter_measure 1000 # ... end end
77. class Foo include GitHub::Backscatter ! def the_method(args) backscatter_measure 1000 # ... end end trace 1 out of every 1000 calls
78. Experiment with confidence. 1 !
79. ! 1
80. ! github/dat-science
81. ! @jbarnette
82. ! @rick
83. !
84. !
85. ! Safely observe alternate code paths.
86. require "dat/science" ! class MyApp::Widget def allows?(user) experiment = Dat::Science::Experiment.new \ "widget-permissions" do |e| ! e.control { old_method } # old way e.candidate { new_method } # new way end ! experiment.run end end
87. require "dat/science" ! class MyApp::Widget def allows?(user) experiment = Dat::Science::Experiment.new \ "widget-permissions" do |e| ! e.control { old_method } # old way e.candidate { new_method } # new way end ! experiment.run end end
88. require "dat/science" ! class MyApp::Widget def allows?(user) experiment = Dat::Science::Experiment.new \ "widget-permissions" do |e| ! e.control { old_method } # old way e.candidate { new_method } # new way end ! experiment.run end end
89. require "dat/science" ! class MyApp::Widget def allows?(user) experiment = Dat::Science::Experiment.new \ "widget-permissions" do |e| ! e.control { old_method } # old way e.candidate { new_method } # new way end ! experiment.run end end
90. require "dat/science" ! class MyApp::Widget def allows?(user) experiment = Dat::Science::Experiment.new \ "widget-permissions" do |e| ! e.control { old_method } # old way e.candidate { new_method } # new way end ! experiment.run end end
91. require "dat/science" ! class MyApp::Widget def allows?(user) experiment = Dat::Science::Experiment.new \ "widget-permissions" do |e| ! e.control { old_method } # old way e.candidate { new_method } # new way end ! experiment.run end end
92. ! Fancy a DSL?
93. require "dat/science" ! class MyApp::Widget def allows?(user) science "widget-permissions" do |e| e.control { model.check_user(user).valid? } e.candidate { user.can? :read, model } end end end
94. require "dat/science" ! class MyApp::Widget def allows?(user) science "widget-permissions" do |e| e.control { model.check_user(user).valid? } e.candidate { user.can? :read, model } end end end
95. ! Create and run.
96. require "dat/science" ! class Blog::Post def permalink(user) science "permalink" do |e| e.control { post[:permalink] } e.candidate { post.slug } e.comparator {|a, b| a.downcase == b.downcase } end end end
97. require "dat/science" ! class Blog::Post def permalink(user) science "permalink" do |e| e.control { post[:permalink] } e.candidate { post.slug } e.comparator {|a, b| a.downcase == b.downcase } end end end
98. ! Ramping up experiments.
99. require "dat/science" ! module Myapp class Experiment < Dat::Science::Experiment def enabled? rand(100) < 10 # Run 10% of the time end end end
100. ! jnunemaker/flipper 2
101. require "dat/science" ! module Myapp class Experiment < Dat::Science::Experiment def enabled? MyApp.flipper[name].enabled? end end end
102. ! Publishing results.
103. require "dat/science" ! module Myapp class Experiment < Dat::Science::Experiment def publish MyApp.instrument "science.#{event}", payload end end end
104. require "dat/science" ! module Myapp class Experiment < Dat::Science::Experiment def publish MyApp.instrument "science.#{event}", payload end end end
107. ! Adding custom context.
108. require "dat/science" ! class Blog::Post def permalink(user) science "permalink" do |e| e.context :post => post e.control { post[:permalink] } e.candidate { post.slug } e.comparator {|a, b| a.downcase == b.downcase } end end end
109. require "dat/science" ! class Blog::Post def permalink(user) science "permalink" do |e| e.context :post => post e.control { post[:permalink] } e.candidate { post.slug } e.comparator {|a, b| a.downcase == b.downcase } end end end
110. Mad science.
111. module GitHub class Experiment < Dat::Science::Experiment class << self # Public: enable mad science mode: returns the candidate values by default # instead of the control. def mad_science=(val) @mad_science = val end ! # Internal: whether or not to always run the candidate instead of the # control. def candidate_instead_of_control? @mad_science end end … end
112. module GitHub class Experiment < Dat::Science::Experiment class << self # Public: enable mad science mode: returns the candidate values by default # instead of the control. def mad_science=(val) @mad_science = val end ! # Internal: whether or not to always run the candidate instead of the # control. def candidate_instead_of_control? @mad_science end end … end
113. ! Analyzing results.
114. ! github/dat-analysis
115. Thanks, Jim.
116. ! EOF