Julia Evans - Systems programming as a swiss army knife

You might think of the Linux kernel as something that only kernel developers need to know about. Not so! It turns out that understanding some basics about kernels and systems programming makes you a better developer, and you can use this knowledge when debugging your normal everyday Python programs.

https://us.pycon.org/2015/schedule/presentation/363/

Transcript

1. SPYING ON YOUR PROGRAMS by Julia Evans Stripe twitter: @b0rk blog: jvns.ca Tweet questions to @b0rk
3. perl | go | c++ | fortran php | python | java | smalltalk INTERCAL | BASIC
4. LINUX-ONLY
5. YOUR PROGRAM = BLACK BOX
6. DEBUGGING: look at the source code add print statements know the programming language
7. DEBUGGING: look at the source code add print statements know the programming language ★★★ be a wizard★★★
9. THIS TALK Wizard school (or, an operating systems primer) Chapter 1: The Case of the Mystery Config File Chapter 2: The Case of the French Website Chapter 3: The Case of the Slow Program
10. WIZARD SCHOOL -OR- WHY YOU SHOULD ❤ YOUR OPERATING SYSTEM
11. WHAT IS AN OPERATING SYSTEM FOR?
12. When I go to http://google.com, kernel code runs for: Typing in the address Handling every network packet Writing history files to disk Allocating memory Communicating with the graphics card
13. HOW TO CALL OPERATING SYSTEM CODE
14. ★★★ SYSTEM CALLS!!! ★★★
15. SYSTEM CALLS: AN OS'S INTERFACE open a file! (o p e n ) start a program! (e x e c v e ) change a file's permissions! (c h m o d )
16. WHAT WE'VE LEARNED Your OS does tons of stuff Programs tell it what to do using system calls
17. USING SYSTEMS KNOWLEDGE TO DEBUG
18. CHAPTER 1: THE CASE OF THE MYSTERY CONFIG FILE
19. Does bash use . b a s h _ p r o f i l e or . b a s h r c ??!??
20. STRACE = WIZARDRY
21. STRACE = TRACING SYSTEM CALLS
22. HOW TO STRACE $ s t r a c e g o o g l e - c h r o m e e x e c v e ( " / u s r / b i n / g o o g l e - c h r o m e " , [ " g o o g l e - c h r o m e " ] , [ / * 5 1 v a r s * / ] ) b r k ( 0 ) = 0 x 1 2 4 f 0 0 0 a c c e s s ( " / e t c / l d . s o . n o h w c a p " , F _ O K ) = - 1 E N O E N T ( N o s u c h f i l e o r
24. OPEN s t r a c e - e o p e n b a s h
25. BASHRC WINS!
26. OTHER AWESOME SYSTEM CALLS w r i t e for log files e x e c v e for starting programs r e c v f r o m for receiving data
27. STRACE ZINE
30. CHAPTER 2: THE CASE OF THE FRENCH WEBSITE
33. ???
34. NETWORK SPYING TO THE RESCUE
35. s u d o n g r e p - d l o 5 0 0 0 i n t e r f a c e : l o ( 1 2 7 . 0 . 0 . 0 / 2 5 5 . 0 . 0 . 0 ) m a t c h : 5 0 0 0 # # # # T 1 2 7 . 0 . 0 . 1 : 4 5 4 3 8 - > 1 2 7 . 0 . 0 . 1 : 5 0 0 0 [ A P ] G E T / H T T P / 1 . 1 . . H o s t : l o c a l h o s t : 5 0 0 0 . . C o n n e c t i o n : k e e p - a l i v e . . C a c h e - C o n t r o l : m a x - a g e = 0 . . A c c e p t : t e x t / h t m l , a p p l i c a t i o n / x h t m l + x m l , a p p l i c a t i o n / x m l ; q = 0 . 9 , i m a g e / w e b p , * / * ; q = 0 . 8 . . U s e r - A g e n t : M o z i l l a / 5 . 0 ( X 1 1 ; L i n u x x 8 6 _ 6 4 ) A p p l e W e b K i t / 5 3 7 . 3 6 ( K H T M L , l i k e G e c k o ) C h r o m e / 4 1 . 0 . 2 2 7 2 . 5 3 S a r i / 5 3 7 . 3 6 . . D N T : 1 . . A c c e p t - E n c o d i n g : g z i p , d e f l a t e , s d c h . . A c c e p t - L a n g u a g e : e n - U S , e n ; q = 0 . 8 . . C o o k i e : u s e r n a m e - l o c a l h o s t - 8 8 8 8 = " 2 | 1 : 0 | 1 0 : 1 4 2 8 4 1 1 8 7 9 | 2 3 : u s e r n a m e - l o c a l h o s t - 8 8 8 8 | 4 8 : M j Y z M T c 2 N G M t Y T A 1 M C 0 0 Y j N k L T k y Y T k t N f a b 7 e e 2 7 9 " . . . . # # # # # # # # # # # # # # # # # # # # # # # T 1 2 7 . 0 . 0 . 1 : 4 5 4 4 0 - > 1 2 7 . 0 . 0 . 1 : 5 0 0 0 [ A P ] G E T / H T T P / 1 . 1 . . U s e r - A g e n t : c u r l / 7 . 2 2 . 0 ( x 8 6 _ 6 4 - p c - l i n u x - g n u ) l i b c u r l / 7 . 2 2 . 0 O p e n S S L / 1 . 0 . 1 z l i b / 1 . 2 . 3 . 4 l i b i d n / 1 . 2 3 l i b r t m p / 2 . 3 . . H o s t : l o c a l h o s t : 5 0 0 0 . . A c c e p t : * / * . . . . # # # # # # # # # # # # # # # # # #
36. A c c e p t - L a n g u a g e : e n - U S
38. NETWORK SPYING TOOLS ngrep tcpdump wireshark mitmproxy
39. CHAPTER 3: THE CASE OF THE SLOW PROGRAM
40. 3 SLOW PROGRAMS 1. CPU time 2. too many writes 3. waiting for a slow server
41. MYSTERY PROGRAM #1
42. $ t i m e p y t h o n m y s t e r y _ 1 . p y 0 . 0 9 u s e r 0 . 0 1 s y s t e m 0 : 0 2 . 1 1 e l a p s e d 5 % C P U
43. WHAT IS IT WAITING FOR?
44. LET'S LOOK INTO THE KERNEL'S SOUL
45. /PROC/P I D /STACK $ p g r e p - f m y s t e r y _ 1 3 1 7 2 8 $ s u d o c a t / p r o c / 3 1 7 2 8 / s t a c k [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 c 0 3 9 > ] s k _ w a i t _ d a t a + 0 x d 9 / 0 x e 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 9 8 b d f > ] t c p _ r e c v m s g + 0 x 6 7 f / 0 x b 5 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 c 1 7 2 b > ] i n e t _ r e c v m s g + 0 x 6 b / 0 x 8 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 7 8 9 5 > ] s o c k _ r e c v m s g + 0 x c 5 / 0 x e 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 7 9 9 e > ] S Y S C _ r e c v f r o m + 0 x e e / 0 x 1 7 0 [ < f f f f f f f f 8 1 6 3 8 7 1 e > ] S y S _ r e c v f r o m + 0 x e / 0 x 1 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 6 6 d > ] s y s t e m _ c a l l _ f a s t p a t h + 0 x 1 a / 0 x 1 f [ < f f f f f f f f f f f f f f f f > ] 0 x f f f f f f f f f f f f f f f f
46. WE WIN! IT WAS THE NETWORK!
47. OUR SERVER @ a p p . r o u t e ( ' / ' ) d e f s l o w ( ) : t i m e . s l e e p ( 2 ) r e t u r n " H i ! " a p p . r u n ( )
48. MYSTERY PROGRAM #2 $ t i m e p y t h o n m y s t e r y _ 2 . p y 2 . 7 4 u s e r 0 . 0 0 s y s t e m 0 : 0 2 . 7 4 e l a p s e d 9 9 % C P U
49. USE A PYTHON PROFILER
50. t o t a l = 0 f o r i i n x r a n g e ( 1 4 0 0 0 0 0 0 ) : t o t a l + = i
51. MYSTERY PROGRAM #3
52. (REALLY A MYSTERY)
53. $ t i m e p y t h o n m y s t e r y _ 3 . p y 0 : 0 2 . 6 1 e l a p s e d 6 2 % C P U $ t i m e p y t h o n m y s t e r y _ 3 . p y 0 : 1 0 . 6 1 e l a p s e d 1 0 % C P U
54. DEMO DEMO
55. WE WIN
56. YOUR PROGRAM = BLACK BOX
57. THERE ARE A LOT OF AWESOME TOOLS
58. LEARN YOUR OPERATING SYSTEM
59. Hacker School Recurse Center
60. THANKS! Julia Evans twitter: @b0rk learn more by reading my blog: http://jvns.ca Come get a strace zine!!!!!