Digging into the Matrix: Practicing Code Archaeology

Most of the code that exists in the world is code you didn’t write, and so most of the code you work with is also code you didn’t write, even if you’re not working with a legacy system. But when we don’t understand that code, we can all too often dismiss, disparage, or even simply delete it - and that’s a mistake!

Almost all the code we interact with was written by other humans, and they wrote the code that way for a reason. If we take the time to understand the context and the story behind a piece of code, we can infer a lot about what those humans meant to achieve, what their culture was like, and what they valued… in other words, we need to become an archaeologist!

Let’s work to build a better understanding of the code we didn’t write, and of who wrote it and why. Using the tools of psychology you can learn what’s happening to your brain when you’re reading someone else’s code, and then learn how you can make understanding it take less cognitive effort. Using the tools of archaeology you can learn how to use your development environment to find the real stories of the people that wrote it, and how that story can protect you from making the same mistakes they did. The ability to read and examine others’ code is essential - you can’t change or improve software without understanding what it already does, so come learn how to do it with skill and compassion!

==Resources==
Gratitude journaling -
https://greatergood.berkeley.edu/article/item/tips_for_keeping_a_gratitude_journal

User roles research -
Just Enough Research, Erika Hall
https://www.nngroup.com/articles/personas-study-guide/
https://jobs-to-be-done.com/jobs-to-be-done-a-framework-for-customer-needs-c883cbf61c90
https://www.nngroup.com/articles/journey-mapping-101/

Reading Code -
The Programmer's Brain, Felienne Hermans

Execution Flow Diagrams -
https://dev.to/conw_y/visualising-execution-flows-59e7

Branching Patterns -
https://martinfowler.com/articles/branching-patterns.html

Parkinson's First Law -
https://en.wikipedia.org/wiki/Parkinson%27s_law

Refactoring -
Emily Bache’s Refactoring Guided Learning, https://www.youtube.com/watch?v=K7xSsNpeM8I
Refactoring, Martin Fowler
Working Effectively with Legacy Code, Michael C Feathers
Refactoring to Patterns, Joshua Kerievsky

Transcript

1. @arthurdoler.com Arthur Doler Resources: Boring details about me are at https://arthurdoler.com Arthur Doler Resources: @arthurdoler.com https://bit.ly/art-digging-into-the-matrix Boring details about me are at https://arthurdoler.com
2. @arthurdoler.com Why hello there.
3. @arthurdoler.com When people ask what I do I always just tell them I solve problems
4. @arthurdoler.com @arthurdoler.com It was dark a lot of the time and there were spiders
5. @arthurdoler.com An artifact buried under centuries of dirt!
6. @arthurdoler.com Even if it seems like it wasn’t, sometimes.
7. @arthurdoler.com OH MY GOD HE’S HERE, WILL HE SIGN MY CITATION?!
8. @arthurdoler.com Use of a whip in a workplace setting not necessarily recommended.
9. @arthurdoler.com Big shoutout to all my Ea-Nasir fans in the audience!
10. @arthurdoler.com The crossover event absolutely nobody expected or wanted
11. @arthurdoler.com @arthurdoler.com Well, this is going to be a problem.
12. @arthurdoler.com @arthurdoler.com Operator, we need an exit now!
13. @arthurdoler.com I thought up this joke about ten seconds after I conceived of this talk.
14. @arthurdoler.com @arthurdoler.com “Failure in pipeline job ‘Build-Pot’ – Boulder consequences engaged”
15. @arthurdoler.com Mindset – Curiosity, Empathy, and Chesterton’s Fence vs the Monkey Ladder Surveying – Exploring the Hourglass Excavation – The History and the Strata Analysis – Writing the Historical Play Progression – What’s Next? How to practice code archaeology at work So let’s crack in!
16. @arthurdoler.com It’s frustrating that there always seems to be work before the work.
17. @arthurdoler.com And a vast middle between the two endpoints
18. @arthurdoler.com This is also the name of my Extremecore Prog Rock band
19. @arthurdoler.com @arthurdoler.com Not necessarily advocating for all of Chesterton’s views here…
20. @arthurdoler.com Imagine being the facilities worker who gets this request
21. @arthurdoler.com Don't you monkey with the monkey
22. @arthurdoler.com Don’t you know you’re gonna / swap the monkey
23. @arthurdoler.com Both views are valid, both views are also likely wrong.
24. @arthurdoler.com Give me fuel, give me fire, give me that which I desire
25. @arthurdoler.com Next refueling station: 1.5 years
26. @arthurdoler.com Who wrote it? Why did they do it? Why did they do it the way they did it? What’s the story of the writing of the code? What’s the story of the code after it was written? If you can get accurate, concrete answers to these, you will own the world
27. @arthurdoler.com With a firehose and a tarantula. Why, how do you cultivate it?
28. @arthurdoler.com “And don’t get me started on binocular vs monocular vision…”
29. @arthurdoler.com Probably triggering for the parents in the audience. Sorry.
30. @arthurdoler.com @arthurdoler.com This little light of mine… I’m gonna let it shine…
31. @arthurdoler.com Not this thing again…
32. @arthurdoler.com Gawronski, 2004
33. @arthurdoler.com @arthurdoler.com Sometimes I get mad about physical architecture, too
34. @arthurdoler.com “Shoshin”, if you were curious about the term
35. @arthurdoler.com “Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.” — Agile Prime Directive Kerth, Project Retrospectives: A Handbook for Team Reviews, 2001
36. @arthurdoler.com Talk about these at parties and watch the fun fail to happen
37. @arthurdoler.com IF YOU WANT TO BOOST YOUR EMPATHY, YOU CAN START WITH GRATITUDE https://greatergood.berkeley.edu/article/item/tips_for_keeping_a_gratitude_journal
38. @arthurdoler.com You’ve got to have pre-built trust between you for this to work well.
39. @arthurdoler.com It’s nice when you stumble on perfect examples for your talk in the wild.
40. @arthurdoler.com Ready for ACTION
41. @arthurdoler.com Oh no, the consequences of MY PAST ACTIONS!
42. @arthurdoler.com Taking an assessment of the current reality
43. @arthurdoler.com Make sure to use trepanation only under the care of a trained professional.
44. @arthurdoler.com YOU ARE GOING TO NEED ABSTRACTIONS Technically you need them for everything, but especially code.
45. @arthurdoler.com These aren’t individual and concrete steps, either.
46. @arthurdoler.com Like sand through the hourglass, these are the days of our system.
47. @arthurdoler.com “They use it because they have to” is still a valid reason.
48. @arthurdoler.com What was it created to achieve? What types of humans use it? What are their roles? What are the goals of each role that uses the system? Wait a sec, this looks familiar…
49. @arthurdoler.com *cookie offer not valid anywhere except Equatorial Guinea.
50. @arthurdoler.com *cue the laugh track*
51. @arthurdoler.com No cookie for you!
52. @arthurdoler.com Just Enough Research, Erika Hall
53. @arthurdoler.com You can skip this step and jump to the next, but it’s a lot harder.
54. @arthurdoler.com What are its parts? Where do they run? What are they written in/with/on? Which parts talk with which other parts? How are those parts built? How are they delivered? More questions than average for one of my talks.
55. @arthurdoler.com Do not think about this picture too long, it’s not good for your brain.
56. @arthurdoler.com This is the step where things will be the clearest.
57. @arthurdoler.com The traditional Ball o’ Mud…
58. @arthurdoler.com … meets the traditional solution for the Gordian Knot.
59. @arthurdoler.com Pick one that’s close to the central reason for the system, if you can.
60. @arthurdoler.com Even figuring the starting place out can be a challenge.
61. @arthurdoler.com @arthurdoler.com “Oh, ah… Just one more thing.”
62. @arthurdoler.com Lesson 1: Chunking Lesson 2: Refactor and document as you go Lesson 3: Replace unfamiliar language construct There are hours worth of content in this book alone!
63. @arthurdoler.com Hermans, 2021
64. @arthurdoler.com Hermans, 2021
65. @arthurdoler.com Hermans, 2021
66. @arthurdoler.com Lesson 1: Chunking Lesson 2: Refactor and document as you go Lesson 3: Replace unfamiliar language constructs Make sure you do the exercises. Don’t just read it.
67. @arthurdoler.com @arthurdoler.com “I don’t even see the code. All I see is box, arrow, box, arrow, cylinder…”
68. @arthurdoler.com Yes, this particular example starts with a custom service runner.
69. @arthurdoler.com When you pay attention to the layers as you move, you see a lot of them.
70. @arthurdoler.com Into the bigger mess
71. @arthurdoler.com Still more exciting than most Family Circus strips.
72. @arthurdoler.com You can represent them as text, but I prefer the visual touch.
73. @arthurdoler.com https://dev.to/conw_y/visualising-execution-flows-59e7
74. @arthurdoler.com And then the biggest mess.
75. @arthurdoler.com If you do get the chance to watch people use your software, TAKE IT!!
76. @arthurdoler.com A good question!
77. @arthurdoler.com Code cannot know its own purpose.
78. @arthurdoler.com @arthurdoler.com There’s a reason tests are sometimes structured with this language.
79. @arthurdoler.com But there’s more to do… we need to go even deeper.
80. @arthurdoler.com Now let’s dig into the past!
81. @arthurdoler.com The colors of the Ancients
82. @arthurdoler.com Hard work and lots of it, like usual.
83. @arthurdoler.com Let’s break this down.
84. @arthurdoler.com It may or may not be on actual scrolls.
85. @arthurdoler.com Documnaeiton of emial trasnactions.
86. @arthurdoler.com The book of Jira, chapter Keno, verse 1123.
87. @arthurdoler.com Collaboratively – (most) software isn’t a solo project.
88. @arthurdoler.com Serving humanity since we managed to invent language.
89. @arthurdoler.com @arthurdoler.com Being a greybeard is a state of mind
90. @arthurdoler.com Check out Active Listening as a toolkit for improving your hearing
91. @arthurdoler.com “Probably shouldn’t ask about the switch to F# from Java, then.”
92. @arthurdoler.com Whatever your particular flavor happens to be
93. @arthurdoler.com @arthurdoler.com Mondays, am I right?! You know I’m right.
94. @arthurdoler.com Look, I do real work sometimes!
95. @arthurdoler.com VISUALIZATION TOOLS CAN LET YOU EASILY EXPLORE THE CHANGES OVER TIME I’m a better developer on the CLI but a better detective with a GUI.
96. @arthurdoler.com https://martinfowler.com/articles/branching-patterns.html
97. @arthurdoler.com Octopus merges are real but they’re not this scary.
98. @arthurdoler.com Hate to be the one to tell you this, but externalities affect your codebase
99. @arthurdoler.com They’re still detectable even once they’re gone.
100. @arthurdoler.com Pay attention to how a tool tells you to use it, and you can see its effects.
101. @arthurdoler.com Trying to build a map for the chaos
102. @arthurdoler.com ASP / VB.NET Backbone / Marionette Javascript / React / Redux Typescript / React / React-Query Wake up, new version of the four humours just dropped
103. @arthurdoler.com This slide suggests there is a Devonian Extinction of frontend tooling
104. @arthurdoler.com My friend, that is too many lenses
105. @arthurdoler.com Taleb, 2007
106. @arthurdoler.com The source here is every other talk I’ve given.
107. @arthurdoler.com Do not waste time trying to find the absolute truth!
108. @arthurdoler.com How much time do you have?
109. @arthurdoler.com A plan for analyzing analysis.
110. @arthurdoler.com yorick.setStatus(“Poor”).alas();
111. @arthurdoler.com You often get the shape of people before you get the details.
112. @arthurdoler.com Claire needs a lot of whiteboards to work.
113. @arthurdoler.com “Get ‘Er Done” Howard, as he’s more commonly known
114. @arthurdoler.com Spoilers: a lot of your opinions on programming are tastes, not facts
115. @arthurdoler.com The mysterious “BA Unknown”
116. @arthurdoler.com Weathervanes – decorating buildings since 48 BCE.
117. @arthurdoler.com @arthurdoler.com You ever have a day at work that feels like this?
118. @arthurdoler.com What resources did the team have? What resources did they lack? What strata is the code from? What now-commonplace solutions and processes didn’t exist then? What were the organizational rules and processes? What other projects were going on? More questions for your dollar than any other talk!
119. @arthurdoler.com We were just banging rocks together back then…
120. @arthurdoler.com Error, no function Compare exists which takes parameters IApple, IOrange
121. @arthurdoler.com The Economist, 1955
122. @arthurdoler.com @arthurdoler.com My main question is where you found a broom in the desert.
123. @arthurdoler.com @arthurdoler.com “Your resume says you’re really good at fishbridges…”
124. @arthurdoler.com WE ALSO CONFUSE FAMILIARITY WITH CORRECTNESS OR RIGHTNESS. Ashcraft, 2006
125. @arthurdoler.com It’s also a valuable process – it saves us cognitive energy.
126. @arthurdoler.com It’s important to
127. @arthurdoler.com It’s important to remain empathetic throughout
128. @arthurdoler.com It’s important to remain empathetic throughout this whole process.
129. @arthurdoler.com Crystal balls usually give answers, not questions, but hey
130. @arthurdoler.com “It’s a Sumerian Hidey-Pot. Common around 2900 BCE.”
131. @arthurdoler.com @arthurdoler.com The ol’ brain’s probably getting a bit crowded at this point…
132. @arthurdoler.com Where do we go from here? Now that all of the children are grown up?
133. @arthurdoler.com You gotta do something to keep busy at work, right?
134. @arthurdoler.com Don’t let the methodology become your end goal.
135. @arthurdoler.com ONCE YOU UNDERSTAND WHY THE CODE IS THE WAY IT IS… IT’S EASIER TO UNDERSTAND HOW TO CHANGE IT! This is it, this is the lesson it’s taken me decades to learn well.
136. @arthurdoler.com If the previous code was constrained, are those constraints gone? If the code was affected by similarity to previous code, should it have been similar? Was the person’s intent incorrect? What strata is the code from? Without all the previous work, you’ll be totally guessing here.
137. @arthurdoler.com Without the story, you’re working blind.
138. @arthurdoler.com Take that, AbstractSingletonFactoryBeanFactory!
139. @arthurdoler.com Refactoring is a surprisingly deep topic and one I enjoy reading about. Emily Bache’s Refactoring Guided Learning – https://www.youtube.com/watch?v=K7xSsNpeM8I Refactoring – Martin Fowler Working Effectively with Legacy Code – Michael C Feathers Refactoring to Patterns – Joshua Kerievsky
140. @arthurdoler.com ALSO – WRITE DOWN WHAT YOU’VE FIGURED OUT!!!!! Not going to let this one go.
141. @arthurdoler.com Or whatever day you’re at work next.
142. @arthurdoler.com “Did they pass out any ink?” “Of course not, don’t be silly.”
143. @arthurdoler.com You’re going to slip up sometimes, but that’s fine.
144. @arthurdoler.com Breaking it down makes it easier to believe you can get it all done, too.
145. @arthurdoler.com @arthurdoler.com I should sell these to add to actual planning poker decks.
146. @arthurdoler.com It’s about time. Whew!
147. @arthurdoler.com Mindset Surveying Extraction Analysis Progression Or “MSEAP” for short, I guess?
148. @arthurdoler.com MINDSET •Remember Chesterton’s Fence and the Monkey Ladder, and start closer to Chesterton’s Fence. •Build your curiosity daily. •Practice empathy towards those who wrote the code. Start with your internals…
149. @arthurdoler.com SURVEYING •Start with one user goal and work through the Hourglass: •The Humans, •The System, •The Code, •…and back up. …move on to the state of the world…
150. @arthurdoler.com EXCAVATION •Find the forces that shaped the code’s construction. •Look through all the histories available to you. •…and opinionated external libraries, too. •Establish strata when the System’s long-lived enough … dig back into the past…
151. @arthurdoler.com ANALYSIS •Flesh out the characters! •Find the backstory! •Then write the play. …find the story…
152. @arthurdoler.com PROGRESSION •Recontextualize based on your own goal •Should you rewrite or just refactor? •Use the story to guide your next steps •Write down what you found …and move forward.
153. @arthurdoler.com We’ve all got a lot of pre-confirming evidence in our priors.
154. @arthurdoler.com @arthurdoler.com I make zero guarantees about boulders
155. @arthurdoler.com The power is yours!
156. @arthurdoler.com Arthur Doler My website is probably still https://arthurdoler.com Resources: THANKS FOR LISTENING! https://bit.ly/art-digging-into-the-matrix