Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Decorators Demystified

Decorators Demystified

Talk given at PyCon Singapore 2015.

Anand Chitipothu

June 18, 2015
Tweet

More Decks by Anand Chitipothu

Other Decks in Technology

Transcript

  1. Decorators @ l o g i n _ r e

    q u i r e d d e f a c c o u n t _ s e t t i n g s ( ) : . . .
  2. Decorators @ r o u t e ( " /

    h e l l o " ) d e f h e l l o ( ) : r e t u r n " h e l l o , w o r l d ! "
  3. Decorators @ d i s k c a c h

    e ( " { 0 } / d i s t r i c t s . t s v " ) d e f d o w n l o a d _ d i s t r i c t s ( s t a t e ) : . . .
  4. Syntactic Sugar! @ m y _ d e c o

    r a t o r d e f f ( ) : p a s s is equivalent to: d e f f ( ) : p a s s f = m y _ d e c o r a t o r ( f )
  5. How does a decorator look like? d e f m

    y _ d e c o r a t o r ( f u n c ) : . . . r e t u r n n e w _ f u n c It is a function It takes a function as argument And returns a new function back
  6. Applying multiple decorators @ d e c o r a

    t o r _ 2 @ d e c o r a t o r _ 1 d e f f ( ) : p a s s is equivalent to: d e f f ( ) : p a s s f = d e c o r a t o r _ 1 ( f ) f = d e c o r a t o r _ 2 ( f )
  7. Decorator with arguments @ r o u t e (

    " / h e l l o " ) d e f h e l l o ( ) : p a s s is equivalent to: d e c o r = r o u t e ( " / h e l l o " ) d e f h e l l o ( ) : p a s s h e l l o = d e c o r ( h e l l o ) route is not the decorator. It is a function that returns the decorator.
  8. Functions x = 1 0 d e f s q

    u a r e ( x ) : r e t u r n x * x > > > s q u a r e ( 4 ) 1 6 > > > s q u a r e < f u n c t i o n s q u a r e a t 0 x 1 0 9 e c 5 1 b 8 >
  9. Functions Functions can be assigned to variable like any other

    data types. x = 1 0 d e f s q u a r e ( x ) : r e t u r n x * x y = x f = s q u a r e > > > f ( 4 ) 1 6 > > > f < f u n c t i o n s q u a r e a t 0 x 1 0 9 e c 5 1 b 8 >
  10. Functions as arguments Functions can be passed as arguments to

    other functions. > > > d e f f s u m ( f , x , y ) : . . . r e t u r n f ( x ) + f ( y ) . . . > > > f s u m ( s q u a r e , 3 , 4 ) 2 5 > > > f s u m ( l e n , " h e l l o " , " s i n g a p o r e " ) 1 4 In fact, there are many standard library functions that take function arguments. > > > m a x ( [ " a l i c e " , " c h a r l i e " , " e v e " , d a v e " , " b o b " ] ) ' e v e ' > > > m a x ( [ " a l i c e " , " c h a r l i e " , " e v e " , d a v e " , " b o b " ] , k e y = l e n ) ' c h a r l i e '
  11. Functions as return values Functions can even return new functions.

    d e f m a k e _ a d d e r ( x ) : d e f a d d ( y ) : r e t u r n x + y r e t u r n a d d a d d 5 = m a k e _ a d d e r ( 5 ) p r i n t a d d 5 ( 2 ) Output: 7
  12. Functions taking variable number of arguments In Python, function can

    take variable number of arguments. > > > m a x ( 3 , 9 , 4 ) 9 > > > m a x ( 3 , 9 , 4 , 2 3 ) 2 3 And here is how they are written. d e f l o g ( l a b e l , * a r g s ) : f o r a i n a r g s : p r i n t l a b e l , a l o g ( " I N F O " , 1 , 2 , 3 ) Output: I N F O 1 I N F O 2 I N F O 3
  13. Functions taking variable number of arguments How about the reverse?

    > > > a r g s = [ 1 , 2 ] > > > l o g ( " I N F O " , * a r g s ) I N F O 1 I N F O 2 > > > a r g s = [ ' D E B U G ' , 1 , 2 ] > > > l o g ( * a r g s ) D E B U G 1 D E B U G 2 This can be used for any function. > > > a r g s = [ ' f f ' , 1 6 ] > > > i n t ( * a r g s ) 2 5 5 > > > i n t ( ' f f ' , 1 6 ) 2 5 5
  14. Function Metadata > > > f r o m t

    i m e i m p o r t a s c t i m e > > > > > > a s c t i m e . _ _ n a m e _ _ ' a s c t i m e ' > > > > > > a s c t i m e . _ _ m o d u l e _ _ ' t i m e ' > > > > > > p r i n t a s c t i m e . _ _ d o c _ _ a s c t i m e ( [ t u p l e ] ) - > s t r i n g C o n v e r t a t i m e t u p l e t o a s t r i n g , e . g . ' S a t J u n 0 6 1 6 : 2 6 : 1 1 1 9 9 8 ' . W h e n t h e t i m e t u p l e i s n o t p r e s e n t , c u r r e n t t i m e a s r e t u r n e d b y l o c a l t i m e ( ) i s u s e d .
  15. Tracing Function Calls (1) Lets say we want to trace

    execution of this code. d e f s q u a r e ( x ) : r e t u r n x * x d e f s u m _ o f _ s q u a r e s ( x , y ) : r e t u r n s q u a r e ( x ) + s q u a r e ( y ) p r i n t s u m _ o f _ s q u a r e ( 3 , 4 )
  16. Tracing Function Calls (2) We can do that with print

    statements. d e f s q u a r e ( x ) : p r i n t " s q u a r e " , x r e t u r n x * x d e f s u m _ o f _ s q u a r e s ( x , y ) : p r i n t " s u m _ o f _ s q u a r e s " , x , y r e t u r n s q u a r e ( x ) + s q u a r e ( y ) p r i n t s u m _ o f _ s q u a r e ( 3 , 4 ) Output: s u m _ o f _ s q u a r e s 3 4 s q u a r e 3 s q u a r e 4 2 5
  17. Tracing Function Calls (3) But can do that without modifying

    those functions using a decorator. d e f t r a c e ( f ) : " " " D e c o r a t o r t o t r a c e c a l l s t o a f u n c t i o n . " " " d e f g ( * a r g s ) : p r i n t f . _ _ n a m e _ _ , a r g s v a l u e = f ( * a r g s ) p r i n t " r e t u r n " , v a l u e r e t u r n v a l u e r e t u r n g This also prints the return value along with function name.
  18. Tracing Function Calls (4) Lets try using it: @ t

    r a c e d e f s q u a r e ( x ) : r e t u r n x * x @ t r a c e d e f s u m _ o f _ s q u a r e s ( x , y ) : r e t u r n s q u a r e ( x ) + s q u a r e ( y ) p r i n t s u m _ o f _ s q u a r e ( 3 , 4 ) Output: s u m _ o f _ s q u a r e s 3 4 s q u a r e 3 r e t u r n 9 s q u a r e 4 r e t u r n 1 6 r e t u r n 2 5 2 5
  19. Tracing Function Calls (6) @ t r a c e

    d e f s q u a r e ( x ) : " " " C o m p u t e s s q u a r e o f a n u m b e r . " " " r e t u r n x * x Lets look at function metadata: > > > s q u a r e < f u n c t i o n g a t 0 x 1 0 4 f d d 2 a 8 > > > > s q u a r e . _ _ n a m e _ _ ' g ' > > > h e l p ( s q u a r e ) H e l p o n f u n c t i o n g i n m o d u l e t r a c e : g ( * a r g s ) What happened to the function metadata?
  20. Tracing Function Calls (7) The f u n c t

    o o l s module provides a helpers to fix that. d e f t r a c e ( f ) : " " " D e c o r a t o r t o t r a c e c a l l s t o a f u n c t i o n . " " " @ f u n c t o o l s . w r a p s ( f ) d e f g ( * a r g s ) : p r i n t f . _ _ n a m e _ _ , a r g s v a l u e = f ( * a r g s ) p r i n t " r e t u r n " , v a l u e r e t u r n v a l u e r e t u r n g
  21. Tracing Function Calls (8) Lets try to pretty print the

    call log. l e v e l = 0 d e f l o g ( t e x t ) : i n d e n t = ' | ' * l e v e l + ' | - - ' p r i n t i n d e n t + t e x t d e f t r a c e ( f ) : d e f g ( * a r g s ) : g l o b a l l e v e l a r g s t r = ' ( ' + " , " . j o i n ( [ r e p r ( a ) f o r a i n a r g s ] ) + ' ) ' l o g ( f . _ _ n a m e _ _ + a r g s t r ) l e v e l + = 1 v a l u e = f ( * a r g s ) l o g ( " r e t u r n " + r e p r ( v a l u e ) ) l e v e l - = 1 r e t u r n v a l u e r e t u r n g
  22. Tracing Function Calls (9) Output: | - - s u

    m _ o f _ s q u a r e s ( 3 , 4 ) | | - - s q u a r e ( 3 ) | | | - - r e t u r n 9 | | - - s q u a r e ( 4 ) | | | - - r e t u r n 1 6 | | - - r e t u r n 2 5 2 5
  23. Tracing Function Calls (10) Lets try tracing fibonacci function. @

    t r a c e d e f f i b ( n ) : i f n = = 0 o r n = = 1 : r e t u r n 1 e l s e : r e t u r n f i b ( n - 1 ) + f i b ( n - 2 )
  24. Tracing Function Calls (11) > > > p r i

    n t f i b ( 4 ) | - - f i b ( 4 ) | | - - f i b ( 3 ) | | | - - f i b ( 2 ) | | | | - - f i b ( 1 ) | | | | | - - r e t u r n 1 | | | | - - f i b ( 0 ) | | | | | - - r e t u r n 1 | | | | - - r e t u r n 2 | | | - - f i b ( 1 ) | | | | - - r e t u r n 1 | | | - - r e t u r n 3 | | - - f i b ( 2 ) | | | - - f i b ( 1 ) | | | | - - r e t u r n 1 | | | - - f i b ( 0 ) | | | | - - r e t u r n 1 | | | - - r e t u r n 2 | | - - r e t u r n 5 5
  25. Memoize (1) How can we get rid of the redundant

    computation in our fib function? How about caching the return values? Doing this is very popular in functional programming world and it is called memoize. d e f m e m o i z e ( f ) : # c a c h e t o s t o r e t h e r e t u r n v a l u e s o f t h e f u n c t i o n c a c h e = { } d e f g ( * a r g s ) : i f a r g s n o t i n c a c h e : c a c h e [ a r g s ] = f ( * a r g s ) r e t u r n c a c h e [ a r g s ] r e t u r n g
  26. Memoize (2) Lets apply memoize to the fib function. @

    m e m o i z e @ t r a c e d e f f i b ( n ) : i f n = = 0 o r n = = 1 : r e t u r n 1 e l s e : r e t u r n f i b ( n - 1 ) + f i b ( n - 2 ) The t r a c e is still kept to see the new execution trace after decorating with memoize.
  27. Memoize (3) > > > f i b ( 4

    ) | - - f i b ( 4 ) | | - - f i b ( 3 ) | | | - - f i b ( 2 ) | | | | - - f i b ( 1 ) | | | | | - - r e t u r n 1 | | | | - - f i b ( 0 ) | | | | | - - r e t u r n 1 | | | | - - r e t u r n 2 | | | - - r e t u r n 3 | | - - r e t u r n 5 5
  28. with_retries (1) Problem: Write a decorator function w i t

    h _ r e t r i e s that continue to retry for 5 times if there is any exception raised in the function. @ w i t h _ r e t r i e s d e f w g e t ( u r l ) : r e t u r n u r l l i b 2 . u r l o p e n ( u r l ) . r e a d ( ) w g e t ( " h t t p : / / g o o g l e . c o m / n o - s u c h - p a g e " ) Should print: w g e t f a i l e d , r e t r y i n g . . . w g e t f a i l e d , r e t r y i n g . . . w g e t f a i l e d , r e t r y i n g . . . w g e t f a i l e d , r e t r y i n g . . . w g e t f a i l e d , r e t r y i n g . . . T r a c e b a c k ( m o s t r e c e n t c a l l l a s t ) : . . . u r l l i b 2 . H T T P E r r o r : H T T P E r r o r 4 0 4 : N o t F o u n d
  29. with_retries (2) i m p o r t f u

    n c t o o l s d e f w i t h _ r e t r i e s ( f ) : @ f u n c t o o l s . w r a p s ( f ) d e f g ( * a r g s ) : f o r i i n r a n g e ( 5 ) : # c a l l t h e o r i g i n a l f u n c t i o n a n d i g n o r e a n y e x c e p t i o n t r y : r e t u r n f ( * a r g s ) e x c e p t : p r i n t f . _ _ n a m e _ _ , " f a i l e d , r e t r y i n g . . . " # c a l l t h e o r i g i n a l f u n c t i o n . # t h i s t i m e , l e t t h e e x c e p t i o n g o o u t . r e t u r n f ( * a r g s ) r e t u r n g
  30. with_retries (3) How to make the number of retries as

    an argument to the decorator? @ w i t h _ r e t r i e s ( n u m _ r e t r i e s = 3 ) d e f w g e t ( u r l ) : r e t u r n u r l l i b 2 . u r l o p e n ( u r l ) . r e a d ( ) even better: @ w i t h _ r e t r i e s ( n u m _ r e t r i e s = 3 , d e l a y = 0 . 1 ) d e f w g e t ( u r l ) : r e t u r n u r l l i b 2 . u r l o p e n ( u r l ) . r e a d ( )
  31. with_retries (4) Remember: w i t h _ r e

    t r i e s is not the decorator now. The return value of w i t h _ r e t r i e s is a decorator. i m p o r t f u n c t o o l s i m p o r t t i m e d e f w i t h _ r e t r i e s ( n u m _ r e t r i e s = 5 , d e l a y = 0 ) : d e f d e c o r a t o r ( f ) : @ f u n c t o o l s . w r a p s ( f ) d e f g ( * a r g s ) : f o r i i n r a n g e ( n u m _ r e t r i e s ) : t r y : r e t u r n f ( * a r g s ) e x c e p t : p r i n t " F a i l e d t o d o w n l o a d , r e t r y i n g . . . " t i m e . d e l a y ( d e l a y ) # C a l l t h e o r i g i n a l f u n c t i o n . T h i s t i m e , l e t t h e e x c e p t i o n g o o u t . r e t u r n f ( * a r g s ) r e t u r n g r e t u r n d e c o r a t o r
  32. fakeweb - the web framework The skeleton of the web

    framework. " " " F a k e w e b f r a m e w o r k . " " " d e f r o u t e ( p a t h ) : " " " R e t u r n s a d e c o r a t o r t o r e g i s t e r m a p p i n g f r o m a p a t h t o a f u n c t i o n . " " " d e f d e c o r ( f ) : # T O D O : F I X M E p a s s r e t u r n d e c o r d e f r e q u e s t ( p a t h ) : " " " R e t u r n s t h e r e s p o n s e r e t u r n e d b y t h e w e b a p p f o r g i v e n p a t h . " " " r e t u r n " 4 0 4 - N o t F o u n d " d e f r u n ( p o r t = 8 0 8 0 ) : " " " R u n s a w e b s e r v e r t o s e r v e t h e w e b a p p . " " " p r i n t " N o t y e t i m p l e m e n t e d "
  33. The web application webapp.py: f r o m f a

    k e w e b i m p o r t r o u t e , r u n @ r o u t e ( " / h e l l o " ) d e f h e l l o ( ) : r e t u r n " H e l l o , w o r l d ! " @ r o u t e ( " / b y e " ) d e f b y e ( ) : r e t u r n " G o o d b y e ! " i f _ _ n a m e _ _ = = " _ _ m a i n _ _ " : r u n ( )
  34. The web client client.py: # i m p o r

    t w e b a p p t o l o a d t h e m a p p i n g i m p o r t w e b a p p f r o m f a k e w e b i m p o r t r e q u e s t i f _ _ n a m e _ _ = = " _ _ m a i n _ _ " : p r i n t r e q u e s t ( " / h e l l o " ) p r i n t r e q u e s t ( " / b y e " ) p r i n t r e q u e s t ( " / n o - s u c h - p a g e " ) Output: 4 0 4 - N o t F o u n d 4 0 4 - N o t F o u n d 4 0 4 - N o t F o u n d
  35. fakeweb - v0.1 _ r o u t e s

    = [ ] d e f r o u t e ( p a t h ) : d e f d e c o r ( f ) : _ r o u t e s . a p p e n d ( ( p a t h , f ) ) r e t u r n f r e t u r n d e c o r d e f r e q u e s t ( p a t h ) : f o r _ p a t h , f u n c i n _ r o u t e s : i f _ p a t h = = p a t h : r e t u r n f u n c ( ) r e t u r n " 4 0 4 - N o t F o u n d "
  36. The Output $ p y t h o n c

    l i e n t . p y H e l l o , w o r l d ! G o o d b y e ! 4 0 4 - N o t F o u n d
  37. Making it real d e f w s g i

    f u n c ( e n v , s t a r t _ r e s p o n s e ) : p a t h = e n v [ ' P A T H _ I N F O ' ] s t a r t _ r e s p o n s e ( ' 2 0 0 O K ' , [ ( " C o n t e n t - t y p e " , " t e x t / p l a i n " ) ] ) r e t u r n r e q u e s t ( p a t h ) d e f r u n ( p o r t = 8 0 8 0 ) : f r o m w s g i r e f . s i m p l e _ s e r v e r i m p o r t m a k e _ s e r v e r s e r v e r = m a k e _ s e r v e r ( " l o c a l h o s t " , p o r t , w s g i f u n c ) p r i n t " h t t p : / / l o c a l h o s t : { } / " . f o r m a t ( p o r t ) s e r v e r . s e r v e _ f o r e v e r ( )
  38. Final Steps Lets run the webapp. $ p y t

    h o n w e b a p p . p y h t t p : / / l o c a l h o s t : 8 0 8 0 / Lets try visiting the URLs. $ c u r l h t t p : / / l o c a l h o s t : 8 0 8 0 / h e l l o H e l l o , w o r l d ! $ c u r l h t t p : / / l o c a l h o s t : 8 0 8 0 / b y e G o o d b y e ! $ c u r l h t t p : / / l o c a l h o s t : 8 0 8 0 / n o - s u c h - p a g e 4 0 4 - N o t F o u n d