Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Julia Evans - Systems programming as a swiss ar...

Julia Evans - Systems programming as a swiss army knife

You might think of the Linux kernel as something that only kernel developers need to know about. Not so! It turns out that understanding some basics about kernels and systems programming makes you a better developer, and you can use this knowledge when debugging your normal everyday Python programs.

https://us.pycon.org/2015/schedule/presentation/363/

PyCon 2015

April 18, 2015
Tweet

More Decks by PyCon 2015

Other Decks in Programming

Transcript

  1. SPYING ON YOUR PROGRAMS by Julia Evans Stripe twitter: @b0rk

    blog: jvns.ca Tweet questions to @b0rk
  2. perl | go | c++ | fortran php | python

    | java | smalltalk INTERCAL | BASIC
  3. DEBUGGING: look at the source code add print statements know

    the programming language ★★★ be a wizard★★★
  4. THIS TALK Wizard school (or, an operating systems primer) Chapter

    1: The Case of the Mystery Config File Chapter 2: The Case of the French Website Chapter 3: The Case of the Slow Program
  5. When I go to http://google.com, kernel code runs for: Typing

    in the address Handling every network packet Writing history files to disk Allocating memory Communicating with the graphics card
  6. SYSTEM CALLS: AN OS'S INTERFACE open a file! (o p

    e n ) start a program! (e x e c v e ) change a file's permissions! (c h m o d )
  7. WHAT WE'VE LEARNED Your OS does tons of stuff Programs

    tell it what to do using system calls
  8. Does bash use . b a s h _ p

    r o f i l e or . b a s h r c ??!??
  9. HOW TO STRACE $ s t r a c e

    g o o g l e - c h r o m e e x e c v e ( " / u s r / b i n / g o o g l e - c h r o m e " , [ " g o o g l e - c h r o m e " ] , [ / * 5 1 v a r s * / ] ) b r k ( 0 ) = 0 x 1 2 4 f 0 0 0 a c c e s s ( " / e t c / l d . s o . n o h w c a p " , F _ O K ) = - 1 E N O E N T ( N o s u c h f i l e o r
  10. OPEN s t r a c e - e o

    p e n b a s h
  11. OTHER AWESOME SYSTEM CALLS w r i t e for

    log files e x e c v e for starting programs r e c v f r o m for receiving data
  12. ???

  13. s u d o n g r e p -

    d l o 5 0 0 0 i n t e r f a c e : l o ( 1 2 7 . 0 . 0 . 0 / 2 5 5 . 0 . 0 . 0 ) m a t c h : 5 0 0 0 # # # # T 1 2 7 . 0 . 0 . 1 : 4 5 4 3 8 - > 1 2 7 . 0 . 0 . 1 : 5 0 0 0 [ A P ] G E T / H T T P / 1 . 1 . . H o s t : l o c a l h o s t : 5 0 0 0 . . C o n n e c t i o n : k e e p - a l i v e . . C a c h e - C o n t r o l : m a x - a g e = 0 . . A c c e p t : t e x t / h t m l , a p p l i c a t i o n / x h t m l + x m l , a p p l i c a t i o n / x m l ; q = 0 . 9 , i m a g e / w e b p , * / * ; q = 0 . 8 . . U s e r - A g e n t : M o z i l l a / 5 . 0 ( X 1 1 ; L i n u x x 8 6 _ 6 4 ) A p p l e W e b K i t / 5 3 7 . 3 6 ( K H T M L , l i k e G e c k o ) C h r o m e / 4 1 . 0 . 2 2 7 2 . 5 3 S a r i / 5 3 7 . 3 6 . . D N T : 1 . . A c c e p t - E n c o d i n g : g z i p , d e f l a t e , s d c h . . A c c e p t - L a n g u a g e : e n - U S , e n ; q = 0 . 8 . . C o o k i e : u s e r n a m e - l o c a l h o s t - 8 8 8 8 = " 2 | 1 : 0 | 1 0 : 1 4 2 8 4 1 1 8 7 9 | 2 3 : u s e r n a m e - l o c a l h o s t - 8 8 8 8 | 4 8 : M j Y z M T c 2 N G M t Y T A 1 M C 0 0 Y j N k L T k y Y T k t N f a b 7 e e 2 7 9 " . . . . # # # # # # # # # # # # # # # # # # # # # # # T 1 2 7 . 0 . 0 . 1 : 4 5 4 4 0 - > 1 2 7 . 0 . 0 . 1 : 5 0 0 0 [ A P ] G E T / H T T P / 1 . 1 . . U s e r - A g e n t : c u r l / 7 . 2 2 . 0 ( x 8 6 _ 6 4 - p c - l i n u x - g n u ) l i b c u r l / 7 . 2 2 . 0 O p e n S S L / 1 . 0 . 1 z l i b / 1 . 2 . 3 . 4 l i b i d n / 1 . 2 3 l i b r t m p / 2 . 3 . . H o s t : l o c a l h o s t : 5 0 0 0 . . A c c e p t : * / * . . . . # # # # # # # # # # # # # # # # # #
  14. A c c e p t - L a n

    g u a g e : e n - U S
  15. 3 SLOW PROGRAMS 1. CPU time 2. too many writes

    3. waiting for a slow server
  16. $ t i m e p y t h o

    n m y s t e r y _ 1 . p y 0 . 0 9 u s e r 0 . 0 1 s y s t e m 0 : 0 2 . 1 1 e l a p s e d 5 % C P U
  17. /PROC/P I D /STACK $ p g r e p

    - f m y s t e r y _ 1 3 1 7 2 8 $ s u d o c a t / p r o c / 3 1 7 2 8 / s t a c k [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 c 0 3 9 > ] s k _ w a i t _ d a t a + 0 x d 9 / 0 x e 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 9 8 b d f > ] t c p _ r e c v m s g + 0 x 6 7 f / 0 x b 5 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 c 1 7 2 b > ] i n e t _ r e c v m s g + 0 x 6 b / 0 x 8 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 7 8 9 5 > ] s o c k _ r e c v m s g + 0 x c 5 / 0 x e 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 7 9 9 e > ] S Y S C _ r e c v f r o m + 0 x e e / 0 x 1 7 0 [ < f f f f f f f f 8 1 6 3 8 7 1 e > ] S y S _ r e c v f r o m + 0 x e / 0 x 1 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 6 6 d > ] s y s t e m _ c a l l _ f a s t p a t h + 0 x 1 a / 0 x 1 f [ < f f f f f f f f f f f f f f f f > ] 0 x f f f f f f f f f f f f f f f f
  18. OUR SERVER @ a p p . r o u

    t e ( ' / ' ) d e f s l o w ( ) : t i m e . s l e e p ( 2 ) r e t u r n " H i ! " a p p . r u n ( )
  19. MYSTERY PROGRAM #2 $ t i m e p y

    t h o n m y s t e r y _ 2 . p y 2 . 7 4 u s e r 0 . 0 0 s y s t e m 0 : 0 2 . 7 4 e l a p s e d 9 9 % C P U
  20. t o t a l = 0 f o r

    i i n x r a n g e ( 1 4 0 0 0 0 0 0 ) : t o t a l + = i
  21. $ t i m e p y t h o

    n m y s t e r y _ 3 . p y 0 : 0 2 . 6 1 e l a p s e d 6 2 % C P U $ t i m e p y t h o n m y s t e r y _ 3 . p y 0 : 1 0 . 6 1 e l a p s e d 1 0 % C P U
  22. THANKS! Julia Evans twitter: @b0rk learn more by reading my

    blog: http://jvns.ca Come get a strace zine!!!!!