Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tweet Emotion: Realtime Tweet Pseudo-Analysis w...

Tomomi Imura
December 05, 2014

Tweet Emotion: Realtime Tweet Pseudo-Analysis with PubNub Data Stream

creating a cartographic visualization using D3.js, Twitter data and the PubNub API.

Tomomi Imura

December 05, 2014
Tweet

More Decks by Tomomi Imura

Other Decks in Technology

Transcript

  1. "How is America feeling now?" Visualized cartographic data of the

    emotional "state" of each state at any given moment 2
  2. Pseudo-analysis Positive or negative sentiments are loosely determined based on

    an analysis of keywords used in tweets p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' m a r v e l o u s ' , ' m a g n i f i c e n t ' , ' f a b u l o u s ' , ' a s t o n i s h i n g ' , ' f a n t a s t i c ' , ' p e a c e f u l ' , ' b r i l l i a n t ' , ' g l o r i o u s ' , ' c h e e r f u l ' , ' g r a c i o u s ' , . . . ] ; 4
  3. Process 1.  using TopoJSON to display a US state map

    using d3.js 2.  using PubNub JavaScript API to consume Twitter stream 3.  colorizing the map by moods 5
  4. GeoJSON •  Open standard format for encoding geographical features •

     Written in JavaScript Object Notation •  Typically large file size 6
  5. TopoJSON •  Extension of GeoJSON that encodes geospatial topology •

     Compressed / Smaller size •  Tailored for only what you need 7
  6. Render TopoJSON with D3.js v a r s v g

    = d 3 . s e l e c t ( ' # m a p ' ) . a p p e n d ( ' s v g ' ) . a t t r ( ' w i d t h ' , 9 0 0 ) . a t t r ( ' h e i g h t ' , 5 5 0 ) ; v a r p a t h = d 3 . g e o . p a t h ( ) . p r o j e c t i o n ( d 3 . g e o . a l b e r s U s a ( ) ) ; 9
  7. Render TopoJSON with D3.js v a r g = s

    v g . a p p e n d ( ' g ' ) ; d 3 . j s o n ( ' u s - s t a t e s . j s o n ' , f u n c t i o n ( e r r o r , t o p o l o g y ) { g . s e l e c t A l l ( ' p a t h ' ) . d a t a ( t o p o j s o n . f e a t u r e ( t o p o l o g y , t o p o l o g y . o b j e c t s . u s S t a t e s ) . f e a t u r e s ) . e n t e r ( ) . a p p e n d ( ' p a t h ' ) . a t t r ( ' c l a s s ' , f u n c t i o n ( d ) { r e t u r n ' s t a t e s ' + d . p r o p e r t i e s . S T A T E _ A B B R ; } ) . a t t r ( ' d ' , p a t h ) . a t t r ( ' f i l l ' , ' g r a y ' ) } ) ; 10
  8. Getting Started with PubNub APIs < s c r i

    p t s r c = " / / c d n . p u b n u b . c o m / p u b n u b . m i n . j s " > < / s c r i p t > v a r c h a n n e l = ' p u b n u b - t w i t t e r ' ; v a r p u b n u b = P U B N U B . i n i t ( { s u b s c r i b e _ k e y : ' s u b - c - 7 8 8 0 6 d d 4 - 4 2 a 6 - 1 1 e 4 - a e d 8 - 0 2 e e 2 d d a b 7 f e ' } ) ; 0 1 . 0 2 . 12
  9. Receiving Live JSON Data To retrieve the live tweets, you

    simply use PubNub s u b s c r i b e ( ) API. p u b n u b . s u b s c r i b e ( { c h a n n e l : c h a n n e l , c a l l b a c k : p r o c e s s D a t a } ) ; 13
  10. Pseudoanalyze Tweets Create a mini dictionary that describes moods v

    a r p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' h a p p y ' , ' m a g n i f i c e n t ' , ' : ) ' , . . . ] ; v a r n e g a t i v e W o r d s = [ ' u n h a p p y ' , ' s o r r y ' , ' a n n o y e d ' , ' d i s l i k e ' , ' a n x i o u s ' , ' : ( ' , . . . ] ; 14
  11. Pseudoanalyze Tweets See if these words are used in each

    tweet, in the callback function f u n c t i o n p r o c e s s D a t a ( d a t a ) { / / l o o k i n g f o r U S d a t a o n l y f o r t h i s d e m o i f ( d a t a . p l a c e . c o u n t r y _ c o d e ! = = ' U S ' ) r e t u r n ; / / C h e c k i f e m o t i o n a l w o r d s a r e u s e d i n t h e t w e e t i f ( p o s i t i v e W o r d s . s o m e ( f u n c t i o n ( v ) { r e t u r n d a t a . t e x t . t o L o w e r C a s e ( ) . i n d e x O f ( v ) > 0 ; } ) ) { d i s p l a y D a t a ( d a t a , p o s i t i v e ) ; } e l s e i f . . . } 15
  12. Pseudoanalyze Tweets Create a mini dictionary that describes moods v

    a r p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' h a p p y ' , ' m a g n i f i c e n t ' , ' : ) ' , . . . ] ; v a r n e g a t i v e W o r d s = [ ' u n h a p p y ' , ' s o r r y ' , ' a n n o y e d ' , ' d i s l i k e ' , ' a n x i o u s ' , ' : ( ' , . . . ] ; 16
  13. Display Data on Map f u n c t i

    o n d i s p l a y D a t a ( d a t a , e m o t i o n ) { v a r s t a t e = d a t a . p l a c e . f u l l _ n a m e . s u b s t r i n g ( c i t y . l a s t I n d e x O f ( ' , ' ) + 1 ) . t r i m ( ) ; i f ( d o c u m e n t . q u e r y S e l e c t o r ( ' . ' + s t a t e ) ) { v a r s t a t e E l = d o c u m e n t . q u e r y S e l e c t o r ( ' . ' + s t a t e ) ; / / c o u n t t a l l y [ s t a t e ] = ( t a l l y [ s t a t e ] | | { p o s i t i v e : 0 , n e g a t i v e : 0 } ) ; t a l l y [ s t a t e ] [ e m o t i o n ] = ( t a l l y [ s t a t e ] [ e m o t i o n ] | | 0 ) + 1 ; . . . 17
  14. ...cont'd / / c o l o r i z

    e s v g p a t h s t a t e E l . s t y l e . f i l l = ( t a l l y [ s t a t e ] . p o s i t i v e > t a l l y [ s t a t e ] . n e g a t i v e ) ? ' p i n k ' : ( ( t a l l y [ s t a t e ] . p o s i t i v e < t a l l y [ s t a t e ] . n e g a t i v e ) ? ' t u r q u o i s e ' : ' b e i g e ' ) ; } } } 18
  15. 19

  16. Thank you! Tomomi Imura @ PubNub  pubnub.com  @pubnub

     @girlie_mac  github.com/pubnub 21