Code as data (RubyConfBY 2019 edition)

Algorithms are typically encoded in code. Sometimes, however, letting non-developers modify algorithms can be beneficial — but for that, we'll have to move the implementation of the algorithm from code into data. Doing so yields a bunch of interesting advantages.

Imagine that you're implementing a complex algorithm that encapsulates some business process at your company. The business stakeholders are pleased, but sometimes come to you with questions, such as

Why is this calculation result so high?
Can you please tweak some factors in the algorithm?

One-off requests like these are probably fine, but occasionally they can be so numerous that you end up spending a significant amount of your time dealing with incoming requests. (You have more interesting stuff to do!) Ideally, business stakeholders themselves would be able to figure out why that calculation result is so high, and would be able to change the factors themselves.

The implementation of the algorithm is in code – and it is typically not feasible to let business stakeholders handle code. (They have more interesting stuff to do!) We can move the algorithm's implementation out of code and into data… and that yields us a bunch of interesting advantages.

Transcript

1. CODE AS DATA DENIS DEFREYNE (PRONOUNS: HE/HIM) @DDFREYNE / DENIS.WS RUBYCONFBY 2019 2019-04-06
2. nanoc / cri / ddmemoize / ddmetrics / …
3. I work at
4. $$$
5. <disclaimer>
6. Why do we need a pricing algorithm?
 
 
 

7. Why do we need a pricing algorithm?
 
 * fixed price?
 

8. Why do we need a pricing algorithm?
 
 * fixed price? unfair.
 

9. Why do we need a pricing algorithm?
 
 * fixed price? unfair.
 * price per hour?

10. Why do we need a pricing algorithm?
 
 * fixed price? unfair.
 * price per hour? hard to calculate.

11. Why do we need a pricing algorithm?
 
 * fixed price? unfair.
 * price per hour? hard to calculate.
 * wait and see what the price turns out to be?
12. Why do we need a pricing algorithm?
 
 * fixed price? unfair.
 * price per hour? hard to calculate.
 * wait and see what the price turns out to be? risky.
13. Why do we need a pricing algorithm?
 
 * fixed price? unfair.
 * price per hour? hard to calculate. * wait and see what the price turns out to be? risky.
 * algorithm?
14. Why do we need a pricing algorithm?
 
 * fixed price? unfair.
 * price per hour? hard to calculate. * wait and see what the price turns out to be? risky.
 * algorithm? yes!
15. $$$
16. $$$ * Business need to know
 why prices are the way they are
 
 
 
 
 

17. $$$ * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 
 

18. $$$ * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 
 

19. $$$ * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 * It is difficult
 to dry-run new formulas
20. def total_price 200 + volume * distance end
21. THE IDEA
22. def total_price 200 + volume * distance end
23. def total_price 200 + volume * distance end sum
24. def total_price 200 + volume * distance end sum product
25. def total_price 200 + volume * distance end sum product variable
26. def total_price 200 + volume * distance end sum product constant variable
28. class SumExpr def initialize(left, right) @left = left @right = right end end
29. class SumExpr def initialize(left, right) @left = left @right = right end end class ProductExpr def initialize(left, right) @left = left @right = right end end
30. class SumExpr def initialize(left, right) @left = left @right = right end end class ProductExpr def initialize(left, right) @left = left @right = right end end class ConstantExpr def initialize(value) @value = value end end

31. class SumExpr def initialize(left, right) @left = left @right = right end end class ProductExpr def initialize(left, right) @left = left @right = right end end class ConstantExpr def initialize(value) @value = value end end
 class VariableExpr def initialize(name) @name = name end end

32. def price_formula SumExpr.new( ConstantExpr.new(200), ProductExpr.new( VariableExpr.new(Tvolume), VariableExpr.new(Tdistance), ) ) end
33. params = { volume: 20, distance: 200, } price_formula.evaluate(params)
34. 
 
 params = { volume: 20, distance: 200, } price_formula.evaluate(params)
 
 4200
35. def evaluate(params)
36. class VariableExpr def initialize(name) @name = name end def evaluate(params) params.fetch(@name) end end
37. class ConstantExpr def initialize(value) @value = value end def evaluate(_params) @value end end
38. class SumExpr def initialize(left, right) @left = left @right = right end def evaluate(params) @left.evaluate(params) + @right.evaluate(params) end end

39. class ProductExpr def initialize(left, right) @left = left @right = right end def evaluate(params) @left.evaluate(params) * @right.evaluate(params) end end

40. 
 
 params = { volume: 20, distance: 200, } 
 

41. 
 
 params = { volume: 20, distance: 200, } price_formula.evaluate(params)
 

42. 
 
 params = { volume: 20, distance: 200, } price_formula.evaluate(params)
 
 4200
43. THE POINT
44. SumExpr.new( ConstantExpr.new(200), ProductExpr.new( VariableExpr.new(Tvolume), VariableExpr.new(Tdistance), ) )
 
 
 
 
 

45. SumExpr.new( ConstantExpr.new(200), ProductExpr.new( VariableExpr.new(Tvolume), VariableExpr.new(Tdistance), ) )
 
 + Z[> 4200 const Z[> 200 * Z[> 4000 var volume Z[> 20 var distance Z[> 200
46. class Node def initialize(name, value, children) @name = name @value = value @children = children end end
47. + Z[> 4200 const Z[> 200 * Z[> 4000 var volume Z[> 20 var distance Z[> 200
48. + Z[> 4200 const Z[> 200 * Z[> 4000 var volume Z[> 20 var distance Z[> 200 Node.new("+", 4200, [const_node, product_node])
49. + Z[> 4200 const Z[> 200 * Z[> 4000 var volume Z[> 20 var distance Z[> 200 Node.new("+", 4200, [const_node, product_node]) Node.new("var volume", 20, [])
51. class VariableExpr def initialize(name) @name = name end def evaluate(params) params.fetch(@name) end end
52. class VariableExpr def initialize(name) @name = name end def evaluate(params) Node.new( "var #{@name}", params.fetch(@name), [], ) end end
53. class ConstantExpr def initialize(value) @value = value end def evaluate(_params) @value end end
54. class ConstantExpr def initialize(value) @value = value end def evaluate(_params) Node.new( 'const', @value, [], ) end end
55. class SumExpr def initialize(left, right) @left = left @right = right end def evaluate(params) left_node = @left.evaluate(params) right_node = @right.evaluate(params) Node.new( '+', left_node.value + right_node.value, [left_node, right_node], ) end end
56. class ProductExpr def initialize(left, right) @left = left @right = right end def evaluate(params) left_node = @left.evaluate(params) right_node = @right.evaluate(params) Node.new( '*', left_node.value * right_node.value, [left_node, right_node], ) end end
57. class Node def to_s # … left to your imagination … end end
58. puts price_formula.evaluate(params)
 
 
 
 
 

59. puts price_formula.evaluate(params) + >?> 4200 const >?> 200 * >?> 4000 var volume >?> 20 var distance >?> 200
60. puts price_formula.evaluate(params).value
 
 
 
 
 

61. puts price_formula.evaluate(params).value
 
 4200
 
 
 

62. THE BEST PART: The price formula did not change!
63. * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 * It is difficult
 to dry-run new formulas
64. * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 * It is difficult
 to dry-run new formulas
65. * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 * It is difficult
 to dry-run new formulas
66. THE POINT (PART II): THE GUI
67. <demozeit>
68. "200 + volume * distance" 
 
 
 
 
 

69. "200 + volume * distance" C> SumExpr.new( ConstantExpr.new(200), ProductExpr.new( VariableExpr.new(Nvolume), VariableExpr.new(Ndistance), ) )
70. parse("200 + volume * distance") C> SumExpr.new( ConstantExpr.new(200), ProductExpr.new( VariableExpr.new(Nvolume), VariableExpr.new(Ndistance), ) )
71. require 'd-parse'
72. module Grammar extend DParseijDSL # … end
73. DIGIT = char_in('0'..'9')
74. CONSTANT_EXPR = repeat1(DIGIT) .capture .map { |d| ConstantExpr.new(d.to_i(10)) }
75. VARIABLE_EXPR = repeat1(char_in('a'..'z')) .capture .map { |d| VariableExpr.new(d) }
76. OPERAND = alt( CONSTANT_EXPR, VARIABLE_EXPR, )
77. OPERATOR = alt( char('+'), char('*'), ).capture
78. OP_SEQ = intersperse( OPERAND, OPERATOR, ).map { |d| OpSeqExpr.new(d) }
79. ROOT = seq( OP_SEQ, eof, ).first
80. input = "200 + volume * distance"
81. input = "200 + volume * distance" raw_expr = GrammarijROOT.apply( input_string, )
82. OpSeqExpr.new( [ ConstantExpr.new(200), "+", VariableExpr.new(Tvolume), "*", VariableExpr.new(Tdistance), ] )
83. def resolve_op_seq_exprs(expr) # … apply shunting yard or so … end
84. resolve_op_seq_exprs(raw_expr)
85. SumExpr.new( ConstantExpr.new(200), ProductExpr.new( VariableExpr.new(Tvolume), VariableExpr.new(Tdistance), ) )
86. input = "200 + volume * distance"
87. input = "200 + volume * distance" GrammarijROOT.apply( input, )
88. input = "200 + volume * distance" resolve_op_seq_exprs( GrammarijROOT.apply( input, ), )
89. input = "200 + volume * distance" price_formula = resolve_op_seq_exprs( GrammarijROOT.apply( input, ), )
90. params = { volume: 20, distance: 200, } 
 

91. params = { volume: 20, distance: 200, } puts price_formula.evaluate(params).value
 

92. params = { volume: 20, distance: 200, } puts price_formula.evaluate(params).value
 
 4200
93. * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 * It is difficult
 to dry-run new formulas
94. * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 * It is difficult
 to dry-run new formulas
95. * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 * It is difficult
 to dry-run new formulas
96. * Business need to know
 why prices are the way they are
 * Developers need to be involved
 to implement new formulas
 * Developers have no good way
 to verify new formulas
 * It is difficult
 to dry-run new formulas
97. THE POINT (PART III): MORE BENEFITS
98. MORE BENEFITS: 
 

99. MORE BENEFITS: * Versioning & locking
 

100. MORE BENEFITS: * Versioning & locking
 * Translate to Ruby 

101. MORE BENEFITS: * Versioning & locking
 * Translate to Ruby * Translate to SQL

102. MORE BENEFITS: * Versioning & locking
 * Translate to Ruby * Translate to SQL
 * Compile with LLVM
103. Is this over-engineering?
104. TAKE-AWAYS
105. * It’s much easier to reason about data
 than to reason about code. 
 

106. * It’s much easier to reason about data
 than to reason about code. * Try novel approaches; they can
 reveal new opportunities.
107. DENIS DEFREYNE @DDFREYNE / DENIS.WS