A Practical Taxonomy of Bugs and How to Squash Them-We Rise 2017

These are the slides I used for We Rise Tech, Women Who Code Atlanta's first conference.

Transcript

1. A COMMON TAXONOMY OF BUGS AND HOW TO SQUASH THEM
2. Instinctual Indications…6 Research Methods…9 Practical Taxonomy…13 Bohrbug…17 Schrödinbug…25 Fractalbug…24 Heisenbug…35 Mandelbug…44 Resources…55 Table of Contents
3. Debugging Skills
4. “As you familiarize yourself with the application, you’ll build up some debugging instincts"
5. “Whenever I see something like this happening, the first thing I do is scan the logs to see if this process is completing or is sending a weird message.”
7. Debugging Instincts “ ”
10. “Whenever I see something like this happening, the first thing I do is scan the logs to see if this process is completing or is sending a weird message.”
11. “Whenever I see #{x}, I always check #{y}”
12. Instincts are just internalized rulesets
13. Research Methods • containment sometimes takes priority over squashing
14. Research Methods • containment sometimes takes priority over squashing • we can only work with facts
15. Research Methods • containment sometimes takes priority over squashing • we can only work with facts • we can’t squash every bug in this talk
16. Observable Attributes
17. Phenetics { ]
18. Warning: Contrived Scenarios Ahead
19. A COMMON TAXONOMY OF BUGS AND HOW TO SQUASH THEM
20. Taxonomy of Bugs Upsettingly Observable Wildly Chaotic {
21. Upsettingly Observable Bugs
22. Wildly Chaotic Bugs
23. How to Squash Them
24. upsettingly observable bug #1 UPSETTINGLY OBSERVABLE
25. Observable Attributes is the bug observable in production?
26. Observable Attributes is the bug observable in production? can it be reproduced locally?
27. Observable Attributes is the bug observable in production? can it be reproduced locally? does it seem to be restricted to one area?
28. Bohrbug deterministic, highly reproducible UPSETTINGLY OBSERVABLE
29. Bohrbug Commonly found in code,sometimes on server UPSETTINGLY OBSERVABLE
30. Bohrbug likes to hide in complex branching in functions, classes or config UPSETTINGLY OBSERVABLE
31. Bohrbug In the wild: validation UPSETTINGLY OBSERVABLE
32. Bohrbug UPSETTINGLY OBSERVABLE Easy to squash
33. Reproduction & Resolution replicate locally and in test UPSETTINGLY OBSERVABLE
34. Reproduction & Resolution replicate locally and in test write the simple solution UPSETTINGLY OBSERVABLE
35. Reproduction & Resolution replicate locally and in test write the simple solution rewrite to be highly readable and extendable UPSETTINGLY OBSERVABLE
36. Bohrbug UPSETTINGLY OBSERVABLE
37. Bohrbug UPSETTINGLY OBSERVABLE
38. upsettingly observable bug #2 UPSETTINGLY OBSERVABLE
39. Observable Attributes how does this work?
40. Observable Attributes how does this work? does this work?
41. Observable Attributes how does this work? does this work? wait, what is this even testing?
42. Observable Attributes how does this work? does this work? wait, what is this even testing? did this ever work?
43. Schrödinbug stick-like body appendages look like twigs UPSETTINGLY OBSERVABLE
44. Schrödinbug Likes to pretend to be working code. On close inspection, reveals itself to be a bug. UPSETTINGLY OBSERVABLE
45. Schrödinbug UPSETTINGLY OBSERVABLE Type I. Type II.
46. Schrödinbug Type I. Code that never worked UPSETTINGLY OBSERVABLE
47. Schrödinbug Type I. Reveal themselves via side-effects UPSETTINGLY OBSERVABLE
48. Schrödinbug Type I. In the wild: UI shows update but database entry not updated. UPSETTINGLY OBSERVABLE
49. Schrödinbug Type II. Code that doesn’t work how you thought UPSETTINGLY OBSERVABLE
50. Schrödinbug Type II. In the wild: Same function being called multiple times UPSETTINGLY OBSERVABLE
51. Basic Reproduction & Resolution replicate locally and in test write the simple solution rewrite to be highly readable and extendable UPSETTINGLY OBSERVABLE
52. -You “How can I reproduce this without knowing exactly what is happening?”
53. Logging as Verification Tool
54. Git Bisect Tool
55. Reproduction & Resolution reproduce the “broken” state locally and in test
56. Reproduction & Resolution reproduce the “broken” state locally and in test add log statements until you can verify what causes the broken state.
57. Reproduction & Resolution reproduce the “broken” state locally and in test add log statements until you can verify what causes the broken state. if the bug did work at some point, find the point at which it did work.
58. Reproduction & Resolution reproduce the “broken” state locally and in test add log statements until you can verify what causes the broken state. if the bug did work at some point, find the point at which it did work. write tests to represent the configuration and flow of the fixed state
59. Schrödinbug UPSETTINGLY OBSERVABLE
60. Schrödinbug UPSETTINGLY OBSERVABLE
61. wildly chaotic bug #1 WILDLY CHAOTIC
62. Observable Attributes Does it appear non-deterministic?
63. Observable Attributes Does it appear non-deterministic? Does it seem to disappear once you observe or debug it?
64. Heisenbug “now you see it, now you don’t” WILDLY CHAOTIC
65. Heisenbug WILDLY CHAOTIC Type I. Type II.
66. Heisenbug Type I. WILDLY CHAOTIC Lives in code
67. Heisenbug Type II. WILDLY CHAOTIC Lives in data
68. -You “How can I reproduce this without testing on production?”
69. Profiling for Verification https://kcachegrind.github.io/html/CallgrindFormat.html Tool
70. FLAME GRAPHS http://www.brendangregg.com/FlameGraphs/cpu-mysql-updated.svg Tool
71. Heisenbug Type I. WILDLY CHAOTIC Profiling can reveal what is being called and when
72. Heisenbug Type II. WILDLY CHAOTIC Profiling can reveal how much time is being spent.
73. Reproduction & Resolution use profiling to find the trigger state
74. Reproduction & Resolution use profiling to find the trigger state use the app (not fixtures or DB manipulation) to get the data in this state
75. Reproduction & Resolution use profiling to find the trigger state use the app (not fixtures or DB manipulation) to get the data in this state recreate that state in test
76. Reproduction & Resolution use profiling to find the trigger state use the app (not fixtures or DB manipulation) to get the data in this state recreate that state in test follow borhbug instruction
77. Heisenbug WILDLY CHAOTIC
78. Heisenbug WILDLY CHAOTIC
79. wildly chaotic bug #2 WILDLY CHAOTIC
80. Observable Attributes is everything broken?
81. Observable Attributes is everything broken? all of it?
82. Observable Attributes is everything broken? all of it? send help??
83. Mandelbug WILDLY CHAOTIC
84. Mandelbug seems like everything is broken at once WILDLY CHAOTIC
85. Mandelbug people are very upset with you WILDLY CHAOTIC
86. Mandelbug likely an issue with your system, not code WILDLY CHAOTIC
87. “The bug is huge and everywhere at once. SQL: could not connect to server: Connection refused was bubbling up all over the place. Jobs won’t run, emails won’t send, every submit button on the site fatal errored.” on-call log WILDLY CHAOTIC
88. Disk Usage Tool df -h
89. Logging as Verification Tool
90. Reproduction & Resolution use df -h to find if all the storage is being use
91. Reproduction & Resolution use df -h to find if all the storage is being use attempt to connect to server & view logs
92. Reproduction & Resolution use df -h to find if all the storage is being used attempt to connect to server & view logs can that be restarted, rotated or killed at this time?
93. Mandelbug WILDLY CHAOTIC
94. Mandelbug WILDLY CHAOTIC
95. A Practical Taxonomy of Bugs Upsettingly Observable Wildly Chaotic { bohrbug schrödinbug mandelbug heisenbug
96. “Debugging Instincts”
97. “Debugging Instincts”
98. Debugging Skills
99. Observe & Classify
100. Verify with logging and time travel
101. Verify without changing state by profiling
102. Use server tools to observe entire process
103. Observe & Classify Verify with logging and time travel Verify without changing state by profiling Use server tools to observe entire process
104. Build Up Your Own Toolkit and Share it
105. Resources & Further Study • “Linux Debugging Tools I Love”, Julia Evans • Systems Performance, Brendan Gregg • “Why Do Computers Stop and What Can Be Done About It?”, Jim Gray • “Debug Patterns for Efficient High- levelSystemC Debugging”, Frank Rogin, Erhard Fehlauer, Christian Haufe, Sebastian Ohnewald