Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Thomas Ballinger - Terminal whispering

Thomas Ballinger - Terminal whispering

Have you ever wanted to add a status bar to your command line program?
Or maybe color the output a bit? Or do you want to write a fullscreen terminal application like ls, top, vim, or emacs? Then you need to speak a bit of terminal! This talk describes how to talk to your terminal from scratch and goes on to show why Python libraries Blessings and Urwid are so awesome.

https://us.pycon.org/2015/schedule/presentation/401/

PyCon 2015

April 18, 2015
Tweet

More Decks by PyCon 2015

Other Decks in Programming

Transcript

  1. a p p l i c a t i o

    n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t e l e t y p e w r i t e r < - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s
  2. a p p l i c a t i o

    n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t y p e w r i t e r < - - - - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s
  3. a p p l i c a t i o

    n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t e l e t y p e w r i t e r < - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s
  4. a p p l i c a t i o

    n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t e l e t y p e w r i t e r < - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s
  5. a p p l i c a t i o

    n - - - - - - - w r i t e ( b y t e s ) Λ | | | r e a d ( ) r e t u r n s b y t e s | | | t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d
  6. a p p l i c a t i o

    n - - - - - - - p r i n t ( ) - - + Λ | | | i n p u t ( ) | | | t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d
  7. What bytes can we write and what happens? a p

    p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) Λ | | | r e a d ( ) r e t u r n s b y t e s | | | t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d
  8. WHAT BYTES CAN WE SEND? ASCII bytes s y s

    . s t d o u t . b u f f e r . w r i t e ( b ' s o m e b y t e s ' ) control characters ([ c h r ( x ) f o r x i n r a n g e ( 3 2 ) ] e.g. ' \ a \ n \ r \ v \ b \ f \ t ' )
  9. i m p o r t t i m e

    , s y s d e f r e p o r t _ p r o g r e s s ( r a t i o , w i d t h = 5 0 ) : f i l l e d = ' = ' * i n t ( r a t i o * w i d t h ) r e s t = ' - ' * ( w i d t h - i n t ( r a t i o * w i d t h ) ) s y s . s t d e r r . w r i t e ( ' \ r | ' + f i l l e d + r e s t + ' | ' ) s y s . s t d e r r . f l u s h ( ) f o r i i n r a n g e ( 1 0 1 ) : r e p o r t _ p r o g r e s s ( i / 1 0 0 . 0 , 5 0 ) t i m e . s l e e p ( . 0 1 )
  10. WHAT BYTES CAN WE SEND? ASCII bytes s y s

    . s t d o u t . b u f f e r . w r i t e ( b ' s o m e b y t e s ' ) control characters ([ c h r ( x ) f o r x i n r a n g e ( 3 2 ) ] or ' \ a \ n \ r \ v \ b \ f \ t ' ) if terminal using an encoding, several bytes to make a character: s y s . s t d o u t . b u f f e r . w r i t e ( b ' \ x e 2 \ x 8 8 \ x 8 2 ' ) s y s . s t d o u t . w r i t e ( u ' ∂ ' ) (via . e n c o d i n g )
  11. WHAT BYTES CAN WE SEND? ASCII bytes s y s

    . s t d o u t . b u f f e r . w r i t e ( b ' s o m e b y t e s ' ) control characters ([ c h r ( x ) f o r x i n r a n g e ( 3 2 ) ] or ' \ a \ n \ v \ b \ f \ t ' ) several bytes may encode a single character: s y s . s t d o u t . b u f f e r . w r i t e ( b ' \ x e 2 \ x 8 8 \ x 8 2 ' ) s y s . s t d o u t . w r i t e ( u ' ∂ ' ) (via . e n c o d i n g ) ANSI escape sequences (' \ x 1 b [ 3 1 m ' )
  12. ANSI ESCAPE SEQUENCES Sequence Meaning \ x 1 b [

    2 3 A move cursor 23 rows up \ x 1 b [ 2 J clear entire screen \ x 1 b [ ? 2 5 l hide the cursor \ x 1 b [ 1 m start writing in bold \ x 1 b [ 3 1 m start writing in red See for more. Wikipedia entry on ANSI escape sequences
  13. e c h o " I n p u t

    p l e a s e " t p u t c u u 1 t p u t c u u 1 t p u t e l r e a d a k e y t p u t c l e a r code sample from http://aplawrence.com/Basics/tput_stty.html
  14. Blessings - Eric Rose - see "Designing Poetic APIs" from

    last year f r o m b l e s s i n g s i m p o r t T e r m i n a l t = T e r m i n a l ( ) p r i n t t . b o l d ( ' H i t h e r e ! ' ) p r i n t t . b o l d _ r e d _ o n _ b r i g h t _ g r e e n ( ' I t h u r t s m y e y e s ! ' ) w i t h t . l o c a t i o n ( 0 , t . h e i g h t - 1 ) : p r i n t ' T h i s i s a t t h e b o t t o m . '
  15. SECRETS UNLOCKED BY WRITING BYTES TO THE TERMINAL Colored and

    styled text (bold, underline, blink...) Moving the cursor anywhere Alternate screen Hide cursor etc. (scroll regions, save/restore cursor position)
  16. What happens when the user types at the keyboard? a

    p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) Λ | | | r e a d ( ) r e t u r n s b y t e s | | | t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d
  17. What happens when the user types at the keyboard? buffering,

    line editing discipline data not passed to application until the user hits return backspace, delete word, reprint line, delete line - not much else (try cat to see this behavior) echoing back characters
  18. a p p l i c a t i o

    n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t e l e t y p e w r i t e r < - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s
  19. + - - - - a p p l i

    c a t i o n - - - - - - - w r i t e ( b y t e s ) | Λ | g e t | | & r e a d ( ) r e t u r n s b y t e s | s e t | | | | | + - - - ⚒ t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d
  20. Turning off echo, using stty $ s t t y

    - e c h o $ e c h o " e c h o i s t u r n e d o f f " $ s t t y e c h o
  21. Turning off echo in Python > > > i m

    p o r t g e t p a s s > > > g e t p a s s . g e t p a s s ( ) P a s s w o r d : ' s e c r e t '
  22. Turning off echo in Python from https://docs.python.org/2/library/termios.html d e f

    g e t p a s s ( p r o m p t = " P a s s w o r d : " ) : i m p o r t t e r m i o s , s y s f d = s y s . s t d i n . f i l e n o ( ) o l d = t e r m i o s . t c g e t a t t r ( f d ) n e w = t e r m i o s . t c g e t a t t r ( f d ) n e w [ 3 ] = n e w [ 3 ] & ~ t e r m i o s . E C H O # l f l a g s t r y : t e r m i o s . t c s e t a t t r ( f d , t e r m i o s . T C S A D R A I N , n e w ) p a s s w d = r a w _ i n p u t ( p r o m p t ) f i n a l l y : t e r m i o s . t c s e t a t t r ( f d , t e r m i o s . T C S A D R A I N , o l d ) r e t u r n p a s s w d
  23. i m p o r t s y s ,

    t e r m i o s c l a s s N o E c h o : d e f _ _ i n i t _ _ ( s e l f , i n _ s t r e a m ) : s e l f . f d = i n _ s t r e a m . f i l e n o ( ) d e f _ _ e n t e r _ _ ( s e l f ) : s e l f . o r i g = t e r m i o s . t c g e t a t t r ( s e l f . f d ) n e w = t e r m i o s . t c g e t a t t r ( s e l f . f d ) n e w [ 3 ] = n e w [ 3 ] & ~ t e r m i o s . E C H O t e r m i o s . t c s e t a t t r ( f d , t e r m i o s . T C S A D R A I N , n e w ) d e f _ _ e x i t _ _ ( s e l f , e x c _ t y p e , e x c _ v a l u e , t r a c e b a c k ) : t e r m i o s . t c s e t a t t r ( s e l f . f d ) , t e r m i o s . T C S A D R A I N , s e l f . o r i g )
  24. w i t h N o E c h o

    ( s y s . s t d i n ) : h = s y s . s t d i n . r e a d l i n e ( ) p r i n t ( h )
  25. SECRET TECHNIQUES MADE POSSIBLE WITH THESE DIALS turn off echo

    send keys immediately current terminal size nonblocking input
  26. What happens when the user types at the keyboard buffering,

    line editing discipline data not passed to application until the user hits return backspace, delete word, reprint line, delete line - not much else to see this, try cat (it doesn't use readline) echoing back characters mapping bytes to keys
  27. EXAMPLE BYTES/KEY MAPPINGS with my particular setup Bytes read Key

    presses b ' a ' a b ' \ x 1 b [ 2 3 ~ ' F11 b ' \ x 1 b [ Z ' Shift-Tab b ' \ x 1 b [ 1 ; 1 0 B ' Meta-Shift-Down b ' \ x 1 b [ 1 ~ ' Home
  28. What happens when the user types at the keyboard buffering,

    line editing discipline data not passed to application until the user hits return backspace, delete word, reprint line, delete line - not much else to see this, try cat (it doesn't use readline) echoing back characters mapping bytes to keys bytes sent without being typed (cursor query)
  29. > > > p r i n t ( "

    \ x 1 b [ 6 n " ) + - - - - a p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) | Λ | g e t | | & r e a d ( ) r e t u r n s b y t e s | s e t | | | | | + - - - ⚒ t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d
  30. > > > p r i n t ( "

    \ x 1 b [ 6 n " )
  31. > > > p r i n t ( "

    \ x 1 b [ 6 n " ) > > > 1 R
  32. > > > p r i n t ( "

    \ x 1 b [ 6 n " ) > > > E S C [ 3 ; 1 R
  33. How can the terminal notify the application? + - -

    - - a p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) | Λ ⚡ | g e t | ⚡ | & r e a d ( ) ⚡s i g n a l s ! | s e t | ⚡ | | | ⚡ | + - - - ⚒ t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d
  34. TOOLS Blessings - Eric Rose - see "Designing Poetic APIs"

    from last year Urwid - Ian Ward - a few examples Click - Armin Ronacher Clint - Kenneth Reitz Python Prompt Toolkit - Jonathan Slenders Curtsies - me!
  35. TOOLS Blessings - Eric Rose - see "Designing Poetic APIs"

    from last year Urwid - Ian Ward - a few examples Click - Armin Ronacher Clint - Kenneth Reitz Curtsies - me! Python Prompt Toolkit - Jonathan Slenders
  36. $ t e l n e t t e r

    m c a s t . o r g https://github.com/doy/python-termcast-server https://github.com/doy/python-termcast-client
  37. FURTHER READING: Terminals, shells, tty, console The TTY Demystified FUN

    VIDEOS http://utcc.utoronto.ca/~cks/space/blog/unix/TypingEOFEffe Loading 4K BASIC with a Teletype Teletype Model 35ASR How The Teleprinter Works- 1940 IMAGE CREDITS Wordcloud credit: https://www.jasondavies.com/wordcloud
  38. THANKS FOR LISTENING! I'll be tweeting terminal tips from @ballingt

    I'll announce when the slides are up on Twitter as well I'll be at the first day of sprints, happy to work on bpython if anyone wants