Brief introduction to Asyncio stack for Python web developers. Describing aiohttp.web, aiopg, and rororo - libraries for building web applications on top of Python 3.
m p o r t a s y n c i o @ a s y n c i o . c o r o u t i n e d e f h e l l o ( ) : r e t u r n " H e l l o , w o r l d ! " l o o p = a s y n c i o . g e t _ e v e n t _ l o o p ( ) l o o p . r u n _ u n t i l _ c o m p l e t e ( h e l l o ( ) ) l o o p . c l o s e ( ) Included in Python 3.4 and later Available in Python 3.3 with $ p i p i n s t a l l a s y n c i o Backported to Python 2.7 as trollius (I don't recommend to use it anyway)
6 . 2 http://aiohttp.readthedocs.org/ i m p o r t a s y n c i o i m p o r t a i o h t t p @ a s y n c i o . c o r o u t i n e d e f f e t c h _ p a g e ( u r l ) : r e s p o n s e = y i e l d f r o m a i o h t t p . r e q u e s t ( ' G E T ' , u r l ) a s s e r t r e s p o n s e . s t a t u s = = 2 0 0 r e t u r n ( y i e l d f r o m r e s p o n s e . r e a d ( ) ) l o o p = a s y n c i o . g e t _ e v e n t _ l o o p ( ) c o n t e n t = l o o p . r u n _ u n t i l _ c o m p l e t e ( f e t c h _ p a g e ( ' h t t p : / / l v i v p y . o r g . u a / ' ) ) p r i n t ( c o n t e n t ) l o o p . c l o s e ( )
r t a s y n c i o f r o m a i o h t t p i m p o r t w e b @ a s y n c i o . c o r o u t i n e d e f h e l l o ( r e q u e s t ) : r e t u r n w e b . R e s p o n s e ( b o d y = ' H e l l o , w o r l d ! ' , c o n t e n t _ t y p e = ' t e x t / p l a i n ' ) a p p = w e b . A p p l i c a t i o n ( ) a p p . r o u t e r . a d d _ r o u t e ( ' G E T ' , ' / ' , h e l l o ) $ g u n i c o r n - k a i o h t t p . w o r k e r . G u n i c o r n W e b W o r k e r - w 9 - t 6 0 a p p : a p p
. 7 . 0 http://aiopg.readthedocs.org/ i m p o r t a s y n c i o f r o m a i o p g i m p o r t c r e a t e _ p o o l @ a s y n c i o . c o r o u t i n e d e f h e l l o ( d s n ) : p o o l = y i e l d f r o m c r e a t e _ p o o l ( d s n ) w i t h ( y i e l d f r o m p o o l . c u r s o r ( ) ) a s c u r s o r : y i e l d f r o m c u r s o r . e x e c u t e ( ' S E L E C T 1 ' ) s e l e c t e d = y i e l d f r o m c u r s o r . f e t c h o n e ( ) a s s e r t s e l e c t e d = = ( 1 , ) l o o p = a s y n c i o . g e t _ e v e n t _ l o o p ( ) l o o p . r u n _ u n t i l _ c o m p l e t e ( h e l l o ( ' d b n a m e = a i o p g u s e r = . . . p a s s w o r d = . . . h o s t = . . . ' ) ) l o o p . c l o s e ( )
i o r e d i s from a i o - l i b s Latest version: 0 . 1 . 5 http://aioredis.readthedocs.org/ a s y n c i o _ r e d i s from third-party developers Latest version: 0 . 1 3 . 4 http://asyncio-redis.readthedocs.org/ Supports PUB/SUB
be a coroutine, and return w e b . R e s p o n s e i m p o r t a s y n c i o f r o m a i o h t t p i m p o r t w e b @ a s y n c i o . c o r o u t i n e d e f i n d e x ( r e q u e s t ) : r e t u r n w e b . R e s p o n s e ( b o d y = ' H e l l o , w o r l d ! ' , c o n t e n t _ t y p e = ' t e x t / p l a i n ' ) It's good idea to put all view functions to v i e w s . p y module
. A p p l i c a t i o n And register handler for a request f r o m a i o h t t p i m p o r t w e b f r o m . i m p o r t v i e w s a p p = w e b . A p p l i c a t i o n ( ) a p p . r o u t e r . a d d _ r o u t e ( ' G E T ' , ' / ' , v i e w s . i n d e x ) Obvious to put application code to a p p . p y module
to use Gunicorn $ g u n i c o r n - b 0 . 0 . 0 . 0 : 8 0 0 0 - k a i o h t t p . w o r k e r . G u n i c o r n W e b W o r k e r - w 9 - t 6 0 p r o j e c t . a p p : a p p Add - - r e l o a d flag to automatically reload Gunicorn server on code change
c i o . c o r o u t i n e d e f s e a r c h ( r e q u e s t ) : " " " S e a r c h b y q u e r y f r o m G E T p a r a m s . " " " q u e r y = r e q u e s t . G E T [ ' q u e r y ' ] l o c a l e = r e q u e s t . G E T . g e t ( ' l o c a l e ' , ' u k _ U A ' ) . . . r e t u r n w e b . R e s p o n s e ( . . . )
c i o . c o r o u t i n e d e f s u b m i t ( r e q u e s t ) : " " " S u b m i t f o r m P O S T d a t a . " " " d a t a = y i e l d f r o m r e q u e s t . p o s t ( ) # N o w P O S T d a t a a v a i l a b l e a s ` ` r e q u e s t . P O S T ` ` . . . r e t u r n w e b . R e s p o n s e ( . . . )
o u t e r . a d d _ r o u t e ( ' G E T ' , ' / s e a r c h ' , v i e w s . s e a r c h ) a p p . r o u t e r . a d d _ r o u t e ( ' P O S T ' , ' / s u b m i t ' , v i e w s . s u b m i t )
c i o . c o r o u t i n e d e f p r o j e c t ( r e q u e s t ) : p r o j e c t _ i d = r e q u e s t . m a t c h _ i n f o [ ' p r o j e c t _ i d ' ] . . . r e t u r n w e b . R e s p o n s e ( . . . )
o u t e r . a d d _ r o u t e ( ' G E T ' , ' / p r o j e c t s / { p r o j e c t _ i d } ' , v i e w s . p r o j e c t ) Or even, a p p . r o u t e r . a d d _ r o u t e ( ' G E T ' , ' / p r o j e c t s / { p r o j e c t _ i d : \ d + } ' , v i e w s . p r o j e c t )
p . r o u t e r . a d d _ r o u t e ( ' G E T ' , ' / p r o j e c t s ' , v i e w s . p r o j e c t s , n a m e = ' p r o j e c t s ' ) a p p . r o u t e r . a d d _ r o u t e ( ' P O S T ' , ' / p r o j e c t s ' , v i e w s . a d d _ p r o j e c t )
s y n c i o . c o r o u t i n e d e f a d d _ p r o j e c t ( r e q u e s t ) : d a t a = y i e l d f r o m r e q u e s t . p o s t ( ) . . . u r l = r e q u e s t . a p p . r o u t e r [ ' p r o j e c t s ' ] . u r l ( ) r e t u r n w e b . H T T P F o u n d ( u r l ) @ a s y n c i o . c o r o u t i n e d e f p r o j e c t s ( r e q u e s t ) : . . .
p i m p o r t a p p Or f r o m f l a s k i m p o r t c u r r e n t _ a p p either Request contains a p p instance for all your needs In app.py, f r o m . i m p o r t s e t t i n g s . . . a p p [ ' s e t t i n g s ' ] = s e t t i n g s
p i m p o r t a p p In views.py, @ a s y n c i o . c o r o u t i n e d e f s e a r c h ( r e q u e s t ) : s e t t i n g s = r e q u e s t . a p p [ ' s e t t i n g s ' ] q u e r y = r e q u e s t . G E T [ ' q u e r y ' ] l o c a l e = r e q u e s t . G E T . g e t ( ' l o c a l e ' , s e t t i n g s . D E F A U L T _ L O C A L E ) . . . r e t u r n w e b . R e s p o n s e ( . . . )
c a t i o n accepts optional middlewares factories sequence Middleware Factory should be a coroutine and returns a coroutine In app.py, @ a s y n c i o . c o r o u t i n e d e f t r i v i a l _ m i d d l e w a r e ( a p p , h a n d l e r ) : @ a s y n c i o . c o r o u t i n e d e f m i d d l e w a r e ( r e q u e s t ) : r e t u r n ( y i e l d f r o m h a n d l e r ( r e q u e s t ) ) r e t u r n m i d d l e w a r e . . . a p p = w e b . A p p l i c a t i o n ( m i d d l e w a r e s = [ t r i v i a l _ m i d d l e w a r e ] )
h t t p _ s e s s i o n Debug Toolbar, a i o h t t p _ d e b u g t o o l b a r In app.py, i m p o r t a i o h t t p _ d e b u g t o o l b a r f r o m a i o h t t p _ d e b u g t o o l b a r i m p o r t t o o l b a r _ m i d d l e w a r e _ f a c t o r y f r o m a i o h t t p _ s e s s i o n i m p o r t s e s s i o n _ m i d d l e w a r e f r o m a i o h t t p _ s e s s i o n . c o o k i e _ s t o r a g e i m p o r t E n c r y p t e d C o o k i e S t o r a g e a p p = w e b . A p p l i c a t i o n ( m i d d l e w a r e s = [ t o o l b a r _ m i d d l e w a r e _ f a c t o r y , s e s s i o n _ m i d d l e w a r e ( E n c r y p t e d C o o k i e S t o r a g e ( b ' 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 ' ) ) ] ) a i o h t t p _ d e b u g t o o l b a r . s e t u p ( a p p )
c i o . c o r o u t i n e d e f e r r o r h a n d l e r _ m i d d l e w a r e ( a p p , h a n d l e r ) : @ a s y n c i o . c o r o u t i n e d e f m i d d l e w a r e ( r e q u e s t ) : t r y : r e t u r n ( y i e l d f r o m h a n d l e r ( r e q u e s t ) ) e x c e p t w e b . H T T P E r r o r a s e r r : # A s i t s p e c i a l c a s e w e c o u l d p a s s ` ` e r r ` ` a s s e c o n d a r g u m e n t t o # e r r o r h a n d l e r r e t u r n ( y i e l d f r o m v i e w s . e r r o r ( r e q u e s t , e r r ) ) r e t u r n m i d d l e w a r e
Supports storing session data in encrypted cookie or redis Enabled by passing s e s s i o n _ m i d d l e w a r e to middleware factories sequence In views.py, f r o m a i o h t t p _ s e s s i o n i m p o r t g e t _ s e s s i o n @ a s y n c i o . c o r o u t i n e d e f l o g i n ( r e q u e s t ) : . . . s e s s i o n = y i e l d f r o m g e t _ s e s s i o n ( r e q u e s t ) s e s s i o n [ ' u s e r _ i d ' ] = u s e r _ i d r e t u r n w e b . R e s p o n s e ( . . . )
Jinja2 Support In app.py, i m p o r t a i o h t t p _ j i n j a 2 i m p o r t j i n j a 2 . . . a i o h t t p _ j i n j a 2 . s e t u p ( a p p , l o a d e r = j i n j a 2 . F i l e S y s t e m L o a d e r ( ' / p a t h / t o / t e m p l a t e s ' ) )
r t a i o h t t p _ j i n j a 2 @ a i o h t t p _ j i n j a 2 . t e m p l a t e ( ' i n d e x . h t m l ' ) d e f i n d e x ( r e q u e s t ) : r e t u r n { ' i s _ i n d e x ' : T r u e }
a i o h t t p _ j i n j a 2 i m p o r t r e n d e r _ t e m p l a t e @ a s y n c i o . c o r o u t i n e d e f i n d e x ( r e q u e s t ) : r e t u r n r e n d e r _ t e m p l a t e ( ' i n d e x . h t m l ' , r e q u e s t , { ' i s _ i n d e x ' : T r u e } )
u j s o n f r o m a i o h t t p i m p o r t w e b d e f j s o n _ r e s p o n s e ( d a t a , * * k w a r g s ) : # S o m e t i m e s u s e r n e e d s t o o v e r r i d e d e f a u l t c o n t e n t t y p e f o r J S O N k w a r g s . s e t d e f a u l t ( ' c o n t e n t _ t y p e ' , ' a p p l i c a t i o n / j s o n ' ) r e t u r n w e b . R e s p o n s e ( u j s o n . d u m p s ( d a t a ) , * * k w a r g s ) Note: I recommend to use ujson for work with JSON data in Python, cause of speed.
t i l s i m p o r t j s o n _ r e s p o n s e @ a s y n c i o . c o r o u t i n e d e f a p i _ b r o w s e r ( r e q u e s t ) : r e t u r n j s o n _ r e s p o n s e ( { ' p r o j e c t s _ u r l ' : r e q u e s t . a p p . r o u t e r [ ' p r o j e c t s ' ] . u r l ( ) , } )
Apache, or other web server for serving static files in production. In app.py, a p p . r o u t e r . a d d _ s t a t i c ( ' / s t a t i c ' , ' / p a t h / t o / s t a t i c ' , n a m e = ' s t a t i c ' )
t t p _ j i n j a 2 . t e m p l a t e ( ' i n d e x . h t m l ' ) d e f i n d e x ( r e q u e s t ) : r e t u r n { ' a p p ' : r e q u e s t . a p p , ' i s _ i n d e x ' : T r u e }
p t s r c = " { { a p p . r o u t e r . s t a t i c . u r l ( f i l e n a m e = " d i s t / j s / p r o j e c t . j s " ) } } " t y p e = " t e x t / j a v a s c r i p t " > < / s c r i p t >
to describe tables with SQLAlchemy and then execute queries Maybe you will miss ORM, but man, ORM is overrated :) Also implements asyncio DBAPI like interface for PostgreSQL
a l c h e m y a s s a m e t a d a t a = s a . M e t a D a t a ( ) u s e r s _ t a b l e = s a . T a b l e ( ' u s e r s ' , m e t a d a t a , s a . C o l u m n ( ' i d ' , s a . I n t e g e r , p r i m a r y _ k e y = T r u e ) , s a . C o l u m n ( ' e m a i l ' , s a . U n i c o d e ( 2 5 5 ) , u n i q u e = T r u e ) , s a . C o l u m n ( ' d i s p l a y _ n a m e ' , s a . U n i c o d e ( 6 4 ) , n u l l a b l e = T r u e ) , )
n c i o f r o m a i o p g . s a i m p o r t c r e a t e _ e n g i n e @ a s y n c i o . c o r o u t i n e d e f g e t _ u s e r ( d s n , u s e r _ i d ) : e n g i n e = y i e l d f r o m c r e a t e _ e n g i n e ( d s n ) w i t h ( y i e l d f r o m e n g i n e ) a s c o n n : q u e r y = u s e r s _ t a b l e . s e l e c t ( u s e r s _ t a b l e . c . i d = = u s e r _ i d ) r e t u r n ( y i e l d f r o m ( y i e l d f r o m c o n n . e x e c u t e ( q u e r y ) ) . f i r s t ( ) ) l o o p = a s y n c i o . g e t _ e v e n t _ l o o p ( ) u s e r = l o o p . r u n _ u n t i l _ c o m p l e t e ( g e t _ u s e r ( ' p o s t g r e s : / / . . . ' , 1 ) ) p r i n t ( u s e r ) l o o p . c l o s e ( )
functions to manipulate data in one place, e.g., s t o r a g e . p y module Put _ t a b l e suffix to every described table, e.g., u s e r s _ t a b l e , not just u s e r s Connect to database in middleware factory Store database connection in main app instance to avoid overflow
c i o . c o r o u t i n e d e f d b _ m i d d l e w a r e ( a p p , h a n d l e r ) : @ a s y n c i o . c o r o u t i n e d e f m i d d l e w a r e ( r e q u e s t ) : d b = a p p . g e t ( ' d b ' ) i f n o t d b : a p p [ ' d b ' ] = d b = y i e l d f r o m c r e a t e _ e n g i n e ( a p p [ ' d s n ' ] ) r e q u e s t . a p p [ ' d b ' ] = d b r e t u r n ( y i e l d f r o m h a n d l e r ( r e q u e s t ) ) r e t u r n m i d d l e w a r e a p p = w e b . A p p l i c a t i o n ( m i d d l e w a r e s = [ d b _ m i d d l e w a r e ] ) a p p [ ' d s n ' ] = ' p o s t g r e s : / / u s e r : p a s s w o r d @ 1 2 7 . 0 . 0 . 1 : 5 4 3 2 / d a t a b a s e ' a p p . r o u t e r . a d d _ r o u t e ( ' G E T ' , ' / u s e r s / { u s e r _ i d : \ d + } ' , v i e w s . u s e r _ p r o f i l e )
s t o r a g e i m p o r t u s e r s _ t a b l e @ a i o h t t p _ j i n j a 2 . t e m p l a t e ( ' u s e r _ p r o f i l e . h t m l ' ) d e f u s e r _ p r o f i l e ( r e q u e s t ) : u s e r _ i d = r e q u e s t . m a t c h _ i n f o [ ' u s e r _ i d ' ] w i t h ( y i e l d f r o m r e q u e s t . a p p [ ' d b ' ] ) a s c o n n : q u e r y = u s e r s _ t a b l e . s e l e c t ( u s e r s _ t a b l e . c . i d = = u s e r _ i d ) u s e r = y i e l d f r o m ( y i e l d f r o m c o n n . e x e c u t e ( q u e r y ) ) . f i r s t ( ) i f n o t u s e r : r a i s e w e b . H T T P N o t F o u n d ( ) r e t u r n { ' u s e r ' : u s e r }
n t e x t l i b i m p o r t c o n t e x t m a n a g e r @ c o n t e x t m a n a g e r d e f a d d _ r o u t e _ c o n t e x t ( a p p , v i e w s , u r l _ p r e f i x = N o n e , n a m e _ p r e f i x = N o n e ) : d e f a d d _ r o u t e ( m e t h o d , u r l , n a m e ) : v i e w = g e t a t t r ( v i e w s , n a m e ) u r l = ( ' / ' . j o i n ( ( u r l _ p r e f i x . r s t r i p ( ' / ' ) , u r l . l s t r i p ( ' / ' ) ) ) i f u r l _ p r e f i x e l s e u r l ) n a m e = ' . ' . j o i n ( ( n a m e _ p r e f i x , n a m e ) ) i f n a m e _ p r e f i x e l s e n a m e r e t u r n a p p . r o u t e r . a d d _ r o u t e ( m e t h o d , u r l , v i e w , n a m e = n a m e ) r e t u r n a d d _ r o u t e
. a p i i m p o r t v i e w s a s a p i _ v i e w s w i t h a d d _ r o u t e _ c o n t e x t ( a p p , a p i _ v i e w s , ' / a p i ' , ' a p i ' ) a s a d d _ r o u t e : a d d _ r o u t e ( ' G E T ' , ' / ' , ' i n d e x ' ) a d d _ r o u t e ( ' G E T ' , ' / p r o j e c t s ' , ' p r o j e c t s ' ) a d d _ r o u t e ( ' P O S T ' , ' / p r o j e c t s ' , ' a d d _ p r o j e c t ' ) a d d _ r o u t e ( ' G E T ' , ' / p r o j e c t / { p r o j e c t _ i d : \ d + } ' , ' p r o j e c t ' ) a d d _ r o u t e ( ' P U T ' , ' / p r o j e c t / { p r o j e c t _ i d : \ d + } ' , ' e d i t _ p r o j e c t ' ) a d d _ r o u t e ( ' D E L E T E ' , ' / p r o j e c t / { p r o j e c t _ i d : \ d + } ' , ' d e l e t e _ p r o j e c t ' )
r t a s y n c i o f r o m . . u t i l s i m p o r t j s o n _ r e s p o n s e @ a s y n c i o . c o r o u t i n e d e f i n d e x ( r e q u e s t ) : r o u t e r = r e q u e s t . a p p . r o u t e r p r o j e c t _ u r l = r o u t e r [ ' a p i . p r o e j c t ' ] . u r l ( p a r t s = { ' p r o j e c t _ i d ' : 4 2 } ) p r o j e c t _ u r l = p r o j e c t _ u r l . r e p l a c e ( ' 4 2 ' , ' { p r o j e c t _ i d } ' ) r e t u r n j s o n _ r e s p o n s e ( { ' u r l s ' : { ' a d d _ p r o j e c t ' : r o u t e r [ ' a p i . a d d _ p r o j e c t ' ] . u r l ( ) , ' p r o j e c t ' : p r o j e c t _ u r l , ' p r o j e c t s ' : r o u t e r [ ' a p i . p r o j e c t s ' ] . u r l ( ) , } } )
n g T y p e P r o x y Settings shouldn't be changed across the app Welcome, rororo.settings In app.py, f r o m r o r o r o . s e t t i n g s i m p o r t i m m u t a b l e _ s e t t i n g s f r o m . i m p o r t s e t t i n g s d e f c r e a t e _ a p p ( * * o p t i o n s ) : s e t t i n g s _ d i c t = i m m u t a b l e _ s e t t i n g s ( s e t t i n g s , * * o p t i o n s ) a p p = w e b . A p p l i c a t i o n ( ) a p p [ ' s e t t i n g s ' ] = s e t t i n g s _ d i c t . . . r e t u r n a p p
c t _ s e t t i n g s s e t u p _ l o c a l e s e t u p _ l o g g i n g s e t u p _ t i m e z o m e t o _ b o o l rororo.logger d e f a u l t _ l o g g i n g _ d i c t u p d a t e _ s e n t r y _ l o g g i n g
A p p l i c a t i o n ( w e b . A p p l i c a t i o n ) : d e f _ _ i n i t _ _ ( s e l f , * * k w a r g s ) : s e l f [ ' s e t t i n g s ' ] = k w a r g s . p o p ( ' s e t t i n g s ' ) s u p e r ( ) . _ _ i n i t _ _ ( * * k w a r g s ) d e f m a k e _ h a n d l e r ( s e l f , * * k w a r g s ) : k w a r g s [ ' d e b u g ' ] = s e l f [ ' s e t t i n g s ' ] [ ' D E B U G ' ] k w a r g s [ ' l o o p ' ] = s e l f . l o o p r e t u r n s e l f . _ h a n d l e r _ f a c t o r y ( s e l f , s e l f . r o u t e r , * * k w a r g s )
JSON Schema for validation Describe Schemas with jsl Welcome, rororo.schemas In api/views.py, f r o m r o r o r o . s c h e m a s i m p o r t S c h e m a f r o m . i m p o r t s c h e m a s @ a s y n c i o . c o r o u t i n e d e f a d d _ p r o j e c t ( r e q u e s t ) : s c h e m a = S c h e m a ( s c h e m a s . a d d _ p r o j e c t , r e s p o n s e _ f a c t o r y = j s o n _ r e s p o n s e ) d a t a = s c h e m a . v a l i d a t e _ r e q u e s t ( ( y i e l d f r o m r e q u e s t . p o s t ( ) ) ) . . . r e t u r n s c h e m a . m a k e _ r e s p o n s e ( p r o j e c t _ d i c t )
s l i m p o r t D o c u m e n t , I n t e g e r F i e l d , S t r i n g F i e l d c l a s s P r o j e c t ( D o c u m e n t ) : n a m e = S t r i n g F i e l d ( m i n _ l e n g t h = 1 , m a x _ l e n g t h = 3 2 , r e q u i r e d = T r u e ) s l u g = S t r i n g F i e l d ( m i n _ l e n g t h = 1 , m a x _ l e n g t h = 3 2 , r e q u i r e d = T r u e ) d e s c r i p t i o n = S t r i n g F i e l d ( m a x _ l e n g t h = 2 5 5 ) c l a s s R e s p o n s e ( P r o j e c t ) : i d = I n t e g e r F i e l d ( m i n i m u m = 0 , r e q u i r e d = T r u e ) r e q u e s t = P r o j e c t . g e t _ s c h e m a ( ) r e s p o n s e = R e s p o n s e . g e t _ s c h e m a ( ) Or use plain Python dicts instead
aiohttp.web 1 working day Quick Notes: Global state -> read all from request Extensions -> ? Blueprints -> list of routes or custom router Flask before/after request -> aiohttp.web middleware Error handlers -> middleware { { u r l _ f o r ( . . . ) } } -> { { a p p . r o u t e r [ . . . ] . u r l ( . . . ) } }
solves my problem with large Flask applications I don't need to use lazy views concept to keep my large apps Pythonic aiohttp.web solves my problem with global state and contexts Everything read from the request. It's just works I'm really tired of Armin's complaints about Python 3
Use Python 3.4 for all good things a i o h t t p . w e b , easy to start and easy to use a i o p g . s a allows you to forget about ORM r o r o r o contains useful helpers for aiohttp.web apps If you miss something, port it to Asyncio stack by yourself Don't forget to payback to Open Source Software