On Notation

My slides for my DataEngConfAU 2022 talk, complete with additional material (FAQs and other supplementary material)

1. Follow @chewxy on Twitter बह#$पते परथमं वाचो अ1ं यत परैरत नामधेय6दधानाः यदेषां शरे;ठं यद=र>मासीA>ेणा तदेषां CनDहतं गुहाGवः On Notation (or YAML MUST DIE)
2. Follow @chewxy on Twitter On Problem Solving
3. Follow @chewxy on Twitter How Do We Solve Problems? 1. Simplify the problem 2. Solve the simplified problem(s) recursively 3. Solve the initial problem
4. Follow @chewxy on Twitter How Do We Solve Problems? 1. Simplify the problem a. Breaking down the problem into component problems b. Model the problem with less resolution 2. Solve the simplified problem(s) recursively 3. Solve the initial problem
5. Follow @chewxy on Twitter How Do We Solve Problems? 1. Simplify the problem a. Breaking down the problem into component problems b. Model the problem with less resolution 2. Solve the simplified problem(s) recursively 3. Solve the initial problem
6. Follow @chewxy on Twitter How Do We Solve Problems? 1. Simplify the problem a. Breaking down the problem into component problems b. Model the problem with less resolution 2. Solve the simplified problem(s) recursively 3. Solve the initial problem a. Combine solutions b. Increase resolution for the solution
7. Follow @chewxy on Twitter How Do We Represent Problems? github.com/owulveryck/wardleytogo
8. Follow @chewxy on Twitter 「白马非马」，可乎？ 曰：可。 曰：何哉？ 曰：马者，所以命形也；白者 所以命色也。命色者非命形也 。故曰：「 白马非马」。 Language: A Unifying Theme
9. Follow @chewxy on Twitter What I’ve Been Thinking About Deep Learning in Go https://gorgonia.org
10. Follow @chewxy on Twitter Neural Network Notations y = σ(Wx + b) yi l = f(Σj wi,j l xi l-1 + bi l) y = f(wi xi + b) y = f ⟨w|x⟩ + |b⟩
11. Follow @chewxy on Twitter Neural Network Notations y = σ(Wx + b) yi l = f(Σj wi,j l xi l-1 + bi l) y = f(wi xi + b) y = f ⟨w|x⟩ + |b⟩
12. Follow @chewxy on Twitter Abstraction, Abstraction, Abstraction* λ As an ML (of the ISWIM kind) - 2017 * just don’t look under the lambda λ>> let x = Matrix(2,2) of float32 WithInit(GlorotN) λ>> let y = Matrix(2,2) of float32 WithInit(Uniform) λ>> let z = mul x y λ>> :∂ z (* manually trigger differentiation of z wrt its inputs *) λ>> (:deriv x) = y (* check that the partial deriv of x is equal to y *) True λ>> let id = fun x -> x in id x ...
13. Follow @chewxy on Twitter As An Algol Descendent (Go-like) - 2018 Commands, Commands Commands λ>> x = Matrix((2,2), Float32, WithInit(GlorotN)) λ>> y = Matrix((2,2), Float32, WithInit(Uniform)) λ>> z = MatMul(x, y) λ>> x.∂ == y // check that the partial deriv of x is equal to y true λ>> func id(x [a]) [a] { return x } id(x): ∀[a]: a → a ...
14. Follow @chewxy on Twitter As An APL - v1 2018 - 2022, v2 2022 life ← {⊃1 ⍵ ∨.∧ 3 4 = +/ +⌿ ¯1 0 1 ∘.⊖ ¯1 0 1 ⌽¨ ⊂⍵}
15. Follow @chewxy on Twitter As An APL - v1 2018 - 2022, v2 2022 x ← ⎕?GN 2 2 y ← ⎕?U 2 2 z ← x +.× y ∂ x = y ⍝ check that partial deriv of x = y 1
16. Follow @chewxy on Twitter INiGo
17. Follow @chewxy on Twitter INiGo INiGo - Iverson Notation in Go
18. Follow @chewxy on Twitter INiGo INiGo - Iverson Notation in Go MoNTOYA - Materializer of Names That Otherwise Yield Aught
19. Follow @chewxy on Twitter INiGo - History • v1 - MNG (2018 - 2022) • v2 - INiGo (current)
20. Follow @chewxy on Twitter INiGo • Slightly different notation from traditional APLs (GNU APL or Dyalog APL) • Tree Calculus-like semantics and a Jay-style shapely type system • More symbols: ◦ ⚠ - Mark function as unsafe ◦ ∇ - Auto differentiation (𝖄 is used for recursion instead) ◦ ∂ - Partial derivative of a value ◦ 🀱 - Horizontal stack ◦ 🁣 - Vertical stack ◦ ⌘ - Command/Escape hatch ◦ ⍼ - Plot ◦ ⎄ - Unpivot
21. Follow @chewxy on Twitter Change The Way You Think
22. Follow @chewxy on Twitter Think different.
23. Follow @chewxy on Twitter +.×
24. Follow @chewxy on Twitter +.× Inner Product
25. Follow @chewxy on Twitter +.×
26. Follow @chewxy on Twitter +.×
27. Follow @chewxy on Twitter +.×
28. Follow @chewxy on Twitter +.×
29. Follow @chewxy on Twitter On + and × 1 + 1 2
30. Follow @chewxy on Twitter On + and × 1 + 1 2 1 2 3 4 × 1 10 100 1000 1 20 300 4000
31. Follow @chewxy on Twitter Side Note: Reduction +/ 1 2 3 4 10
32. Follow @chewxy on Twitter +.×
33. Follow @chewxy on Twitter +.× a b c d e f
34. Follow @chewxy on Twitter +.× a b c d e f
35. Follow @chewxy on Twitter +.× a b c d e f a×d
36. Follow @chewxy on Twitter +.× a b c d e f a×d b×e
37. Follow @chewxy on Twitter +.× a b c d e f a×d b×e c×f
38. Follow @chewxy on Twitter +.× a b c d e f a×d b×e c×f
39. Follow @chewxy on Twitter +.× a b c d e f a×d + b×e + c×f
40. Follow @chewxy on Twitter +.×
41. Follow @chewxy on Twitter +.×
42. Follow @chewxy on Twitter +.×
43. Follow @chewxy on Twitter +.×
44. Follow @chewxy on Twitter +.×
45. Follow @chewxy on Twitter ⋀.⋀
46. Follow @chewxy on Twitter 1 2 3 4 +.× 5 6 7 8 70
47. Follow @chewxy on Twitter 1 2 3 4 +.× 5 6 7 8 70 1 2 3 4 ,.× 5 6 7 8 5 12 21 32
48. Follow @chewxy on Twitter 1 2 3 4 +.× 5 6 7 8 70 1 2 3 4 ,.× 5 6 7 8 5 12 21 32 1 2 3 4 ,.{⍺'×'⍵','} 5 6 7 8 1 × 5 , 2 × 6 , 3 × 7 , 4 × 8 ,
49. Follow @chewxy on Twitter Beautiful APL c (×÷+.×) p
50. Follow @chewxy on Twitter cp ← {({(⍳∘(⌈/⍵))⍵}|+\(⍵-(+/÷≢)⍵))⌷⍺} Real Life APL
51. Follow @chewxy on Twitter Things I Discovered By Virtue of New Notation • Rediscovery of differentiation by means of skolemization of dual numbers. • S(·) → + → × → ✳ • Various isomorphisms that may be useful for developing neural networks.
52. Follow @chewxy on Twitter On Problems (Redux)
53. Follow @chewxy on Twitter github.com/chewxy/annotation-mode.el
54. Follow @chewxy on Twitter Programming in the large vs Programming in the small
55. Follow @chewxy on Twitter “Software engineering is programming, integrated over time” - Titus Winters
56. Follow @chewxy on Twitter
57. Follow @chewxy on Twitter On Good Notation(s)
58. Follow @chewxy on Twitter
59. Follow @chewxy on Twitter The Properties of a Good Notation • Ease of Expressing Constructs in Problem Solving • Suggestivity • Subordination of Detail • Economy • Amenable to Formal Proofs
60. Follow @chewxy on Twitter The Properties of a Good Notation • Ease of Expressing Constructs in Problem Solving • Suggestivity • Subordination of Detail • Economy • Amenable to Formal Proofs
61. Follow @chewxy on Twitter The Properties of a Good Notation • Ease of Expressing Constructs in Problem Solving • Suggestivity • Subordination of Detail • Economy • Amenable to Formal Proofs
62. Follow @chewxy on Twitter The Properties of a Good Notation • Ease of Expressing Constructs in Problem Solving • Suggestivity • Subordination of Detail • Economy • Amenable to Formal Proofs
63. Follow @chewxy on Twitter Multiple Notation In Use • Specification and implementation ◦ Type language vs term language ◦ Googlers Protobuf transformers
64. Follow @chewxy on Twitter Multiple Notation In Use • Specification and implementation ◦ Type language vs term language ◦ Googlers Protobuf transformers • Mixed computational models ◦ Regular expressions ◦ SQL
65. Follow @chewxy on Twitter Multiple Notation In Use • Specification and implementation ◦ Type language vs term language ◦ Googlers Protobuf transformers • Mixed computational models ◦ Regular expressions ◦ SQL • Hierarchical use ◦ Functional core, imperative shell ◦ Imperative core, functional BAU (e.g. NaughtyDog’s Game Oriented Assembly Lisp )
66. Follow @chewxy on Twitter Multiple Notation In Use • Specification and implementation ◦ Type language vs term language ◦ Googlers Protobuf transformers • Mixed computational models ◦ Regular expressions ◦ SQL • Hierarchical use ◦ Functional core, imperative shell ◦ Imperative core, functional BAU (e.g. NaughtyDog’s GOAL - OpenGOAL.dev) • DSLs and Mixins • Intensional and extensional views w/ periodic unification • Multi-surface languages (e.g. Racket, JetBrains MPS, etc)
67. Follow @chewxy on Twitter Poly(glot) Pleasure “A change in perspective is worth 80 IQ points” - Alan Kay
68. Follow @chewxy on Twitter Properties of Good Notations • Good interoperation/integration • Fit for purpose • Distinctive
69. Follow @chewxy on Twitter Good Interop Gold standard: 1 + 1
70. Follow @chewxy on Twitter Good Interop Gold standard: 1 + 1 💩 standard: λm.λn.λf.λx.mf(nfx) λf.λx.fx λf.λx.fx
71. Follow @chewxy on Twitter Good Interop Gold standard: 1 + 1 Whelmed standard: import arith .. arith.add(arith.one, arith.one) 💩 standard: λm.λn.λf.λx.mf(nfx) λf.λx.fx λf.λx.fx
72. Follow @chewxy on Twitter Properties of Good Notations • Good interoperation/integration • Fit for purpose • Distinctive
73. Follow @chewxy on Twitter “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” - Albert Einstein
74. Follow @chewxy on Twitter Properties of Good Notations • Good interoperation/integration • Fit for purpose • Distinctive
75. Follow @chewxy on Twitter Reality Knocks On My Door…
76. Follow @chewxy on Twitter Recommendations • Invent specific languages to solve your specific problem. • Invent multiple languages to solve one problem. • Take a hundred metre view: ◦ Survey similarities ◦ Examine exceptions ◦ Understand unifiers
77. Follow @chewxy on Twitter Alternatives Cue/Dhall instead of YAML Protobuf instead of json dbt for “dependent” type checking of databases Malloy for “modular” SQL Nix for reproducible builds
78. Follow @chewxy on Twitter Alternatives Cue/Dhall instead of YAML Protobuf instead of json dbt for “dependent” type checking of databases Malloy for “modular” SQL Nix for reproducible builds Deterministic Name Algebra }
79. Follow @chewxy on Twitter Takeaways
80. Follow @chewxy on Twitter Takeaways • Language := rules + elements that the rules work on. • Solving problems := establishing rules/constraints on a bunch of things. • Notation := how language is written. • Changing the way we write changes the way we solve problems. • Use more different notations to help us think through and solve problems. • Don’t add more cruft to the already fragile ecosystem.
81. Follow @chewxy on Twitter @chewxy on Twitter Thank You Mei Lee Gary Miller Sarah Catanzaro Dane Laban Gajan Suthokumar Adam Burgess Daniel Owji Darrell Chua Gareth Seneque
82. Follow @chewxy on Twitter Appendix
83. Follow @chewxy on Twitter FAQs - On INiGo’s Creation • Why not lisp/Have you heard of Racket? ◦ (((o(°▽°)o))) • Why not Rust? ◦ People keep comparing Rust to Go. They should really compare Rust with chess. Both are tactical games. Go is more of a strategy game. • Why not Haskell/Ocaml? ◦ Both are pure functional programming languages. Ocaml is pure, and Haskell is extra virgin. I can’t afford the purity because I have a day job. • Why don’t you use Dyalog APL? ◦ I don’t understand their business model, which appears to be stuck 20 years in the past. Plus, I control my own language.
84. Follow @chewxy on Twitter FAQs - On Use of INiGo and APLs • How do you type INiGo or APL code? ◦ With a keyboard. Specifically, all the keyboards I use either have QMK or ZMK as a firmware. The QMK keyboards have a special APL layer, which allows direct input. • Are the symbols necessary? ◦ Yes and no. They may be replaced by words or any other combination of symbols on a keyboard (like J-lang does). But I feel the symbols are especially well thought out and are suggestive of what they do. ◦ e.g ⍉ is transpose and ⌽ is reverse/rotate along the Y axis. These give visual cues as to what you’re doing.
85. Follow @chewxy on Twitter FAQs • What is the Sanskrit text background in the first slide? ◦ Book 10 of the Rig Veda, on knowledge ◦ The verse reads: “Bṛhaspati, when the wise established name-giving - the first principles of Vāc (Language) - all that was excellent and pure was brought forth to light” • What is the Chinese text in the background of the Language slide? ◦ The White Horse Dialogue by Gongsun Long (320-250 B.C) ◦ The words observes a linguistic paradox - that a white horse is not a horse ◦ The crux of the paradox is summed up by Bill Clinton: “that depends on what your definition of ‘is’ is”.
86. Follow @chewxy on Twitter FAQs - on Representing AI-ish Things • Have you heard of Dyna? Yes. I have also used it. It’s a declarative language for maths that I feel is better than traditional mathematical language. • The mathematical notations on neural networks aren’t all equally showing the same information! First, that’s not a question. And yes, I am aware of that. Surely I get to bias things a little to make my point.
87. Follow @chewxy on Twitter FAQs • What is with the scary head picture on the ALGOL slide? ◦ Algol is known as the demon star in the constellation of Perseus. ◦ Perseus killed Medusa, a Gorgon. The demon star refers to Medusa. ◦ Gorgonia was named after the Gorgons.
88. Follow @chewxy on Twitter FAQs on Multiple Notation • Didn’t we already go through this with the mess of UML and the like in the 90s? You want us to go back to writing specs like that? Yes. We lacked wisdom then. We are wiser now. Knowing when to use things like UML for problem solving and when to use it for specification is wisdom.
89. Follow @chewxy on Twitter Supplementary Material
90. Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change ✳ * *
91. Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Traditionally: • Repeated S ⇒ + • Repeated + ⇒ × • Repeated × ⇒ *
92. Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Traditionally: • Repeated S ⇒ + • Repeated + ⇒ × • Repeated × ⇒ * But: ab ≠ ba Compiler writers love the commutativity trick!
93. Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity?
94. Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity? a ✳ b = aln b
95. Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity? a ✳ b = aln b ✓ (a ✳ b) ✳ c = a ✳ (b ✳ c) ✓ a ✳ b = b ✳ a ✓ a ✳ (b × c) = a ✳ b × a ✳ c Identity: e Inverse Identity: e1/ln(a)
96. Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity? a ✳ b = aln b ✓ (a ✳ b) ✳ c = a ✳ (b ✳ c) ✓ a ✳ b = b ✳ a ✓ a ✳ (b × c) = a ✳ b × a ✳ c Identity: e Inverse Identity: e1/ln(a) a, b, c > 0