Refactor Like a Scientist

Tests are a good baseline for defining the behavior of a program,
but for complicated programs, are tests enough?
[Scientist](https://github.com/trello/scientist) is a Node.js library
for creating and running experiments between old and new pieces of code.
Scientist makes it easy to refactor critical code paths with confidence.
It safely exposes refactorings to actual production interactions and measures
potential mismatches. Part experience report, refactoring discussion,
and workshop, this talk makes the case for changing software safely and will also explore safe refactoring techniques
at companies like Trello and GitHub.

Transcript

1. Refactor Like a Scientist Fred Galoso github.com/fredrick @wayoutmind
8. Change is inevitable. Stuff happens blows up.
9. Why does software change? Scale Requirements Maintenance
10. Fix it. How does it work now? Apply changes. Don’t break anything. Good luck!
11. "List of Software Bugs"
12. The year 2000 problem spawned fears of worldwide economic collapse and an industry of consultants providing last-minute fixes.
13. A bug in the code controlling the Therac-25 radiation therapy machine was directly responsible for at least five patient deaths in the 1980s when it administered excessive quantities of X-rays.
14. The Northeast blackout of 2003 was triggered by a local outage that went undetected due to a race condition in General Electric Energy's XA/21 monitoring software.
15. Smart ship USS Yorktown was left dead in the water in 1997 for nearly 3 hours after a divide by zero error.[
16. In January 2009, Google's search engine erroneously notified users that every web site worldwide was potentially malicious, including its own.
17. How do we safely change software?
18. How do we safely refactor software?
19. Tests Code Review Specifications Quality Assurance Safe/Continuous Deployment Practices … Oh my!
22. –Dr. Edsger W. Dijkstra “Program testing can be used to show the presence of bugs, but never to show their absence”
23. Since software has bugs, given enough time and volume, your data will have bugs, too. –Jesse Toth, GitHub
24. GitHub Permissions Refactor
25. class Repository def pullable_by?(user) self.is_collaborator?(user) end end
26. Measure Twice, Cut Over Once
27. Scientific Method Question Research ☝ Hypothesis Experiment Analysis ✍ Communication
28. Scientist (Ruby) http://githubengineering.com/scientist/
29. def pullable_by?(user) science "repository.pullable-by" do |experiment| experiment.context :repo => id, :user => user.id experiment.use { is_collaborator?(user) } experiment.try { has_access?(user) } end end
33. –Jeff Atwood Atwood's Law: any application that can be written in JavaScript, will eventually be written in JavaScript
34. Scientist (Node.js) npm install scientist https://github.com/trello/scientist
35. Example git clone https://github.com/fredrick/refactor-like-a-scientist.git cd refactor-like-a-scientist/examples npm install node —-harmony https://github.com/fredrick/refactor-like-a-scientist
36. Array Sum 'use strict'; const sum = (arr) => { let sum = 0; for (var i of arr) { sum += i; } return sum; }; module.exports = sum;
37. 'use strict'; const science = require('scientist/console'); const iterative = (arr) => { let sum = 0; for (var i of arr) { sum += i; } return sum; }; const functional = (arr) => { return arr.reduce((a, b) => { return a + b }); }; const sum = (arr) => { return science('sum', (experiment) => { experiment.use(() => iterative(arr)); experiment.try(() => functional(arr)); }); }; module.exports = sum;
38. > const sum = require('./sum-science') undefined > sum([1, 2, 3]) 6 > Experiment candidate matched the control { context: {}, result: 'value: 6' }
39. https://sticky-venom.glitch.me/ Logs/code https://glitch.com/edit/#!/sticky-venom
40. Side effects?!
41. Side effects?! API calls
42. Side effects?! Data API calls
43. Side effects?! Data Persistence API calls
44. Side effects?! Data Persistence I/O API calls
45. 1. Isolation 2. Instrumentation 3. Side effect
46. function processUser(user) { # Isolation const oldDateCalculation = complexDateCreator.create(); const newDateCalculation = new Date(); # Instrumentation const dateProcessed = science('datedProcessed', (experiment) => { experiment.use(() => oldDateCalculation); experiment.try(() => newDateCalculation); }); # Side effect user.lastProcessed = dateProcessed; user.save(); return user; }
47. function getUserById(userId) { const user = science('getUserById', (experiment) => { experiment.use(() => Users.findById(userId)); experiment.try(() => { SQL(‘SELECT * FROM Users WHERE id = %s’, userId)); } }); return user; }
48. function applyBrakes(userId) { const brakeThreshold = science(‘brakeThreshold', (experiment) => { experiment.use(() => ExistingProximityRadar.getBrakeThreshold()); experiment.try(() => NewHybridComputerVisionAndRadar.getBrakeThreshold()); }); return activateBrakes(brakeThreshold); }
49. Isolation |— /—| Side effect Separation is hard.
50. Loose coupling is even harder.
51. Challenges
52. Duplicate code paths can be expensive.
53. The conditions for exposure between candidate and control flows can be different.
54. Maintaining old and new code together in the face of change.
55. Experimenting on third party dependencies can be difficult.
56. Pragmatism
57. Identify and start experimenting around small portions of mission critical functionality.
58. Incremental > Wholesale changes
59. Incremental > Wholesale experimentation
60. Instrument everything
61. Safety is worth it and necessary!
62. Scientific Tools Scientist4J (Java) PHP Ruby Scientist.net (.NET) JavaScript (Node.js)
63. Async Sampling Skipping Metrics Customization … https://github.com/trello/scientist Fred Galoso github.com/fredrick @wayoutmind