-", "He kindly stopped for me -", "The Carriage held but just Ourselves -", "and Immortality") > > text ## [1] "Because I could not stop for Death -" "He kindly stopped for me -" ## [3] "The Carriage held but just Ourselves -" "and Immortality" What do we mean by tidy text?
tibble: 20 × 2 ## line word ## <int> <chr> ## 1 1 because ## 2 1 i ## 3 1 could ## 4 1 not ## 5 1 stop ## 6 1 for ## 7 1 death ## 8 2 he ## 9 2 kindly ## 10 2 stopped ## # ... with 10 more rows • Other columns have been retained • Punctuation has been stripped • Words have been converted to lowercase What do we mean by tidy text?
%>% unnest_tokens(word, text) > > tidy_books # A tibble: 725,054 × 4 book linenumber chapter word <fctr> <int> <int> <chr> 1 Sense & Sensibility 1 0 sense 2 Sense & Sensibility 1 0 and 3 Sense & Sensibility 1 0 sensibility 4 Sense & Sensibility 3 0 by 5 Sense & Sensibility 3 0 jane 6 Sense & Sensibility 3 0 austen 7 Sense & Sensibility 5 0 1811 8 Sense & Sensibility 10 1 chapter 9 Sense & Sensibility 10 1 1 10 Sense & Sensibility 13 1 the # ... with 725,044 more rows
word sentiment n <chr> <chr> <int> 1 miss negative 1855 2 well positive 1523 3 good positive 1380 4 great positive 981 5 like positive 725 6 better positive 639 7 enough positive 613 8 happy positive 534 9 love positive 495 10 pleasure positive 462 # ... with 2,575 more rows Which words contribute to each sentiment?
n total <fctr> <chr> <int> <int> 1 Mansfield Park the 6206 160460 2 Mansfield Park to 5475 160460 3 Mansfield Park and 5438 160460 4 Emma to 5239 160996 5 Emma the 5201 160996 6 Emma and 4896 160996 7 Mansfield Park of 4778 160460 8 Pride & Prejudice the 4331 122204 9 Emma of 4291 160996 10 Pride & Prejudice to 4162 122204 # ... with 40,369 more rows TF-IDF
book_words # A tibble: 40,379 × 7 book word n total tf idf tf_idf <fctr> <chr> <int> <int> <dbl> <dbl> <dbl> 1 Mansfield Park the 6206 160460 0.03867631 0 0 2 Mansfield Park to 5475 160460 0.03412065 0 0 3 Mansfield Park and 5438 160460 0.03389007 0 0 4 Emma to 5239 160996 0.03254118 0 0 5 Emma the 5201 160996 0.03230515 0 0 6 Emma and 4896 160996 0.03041069 0 0 7 Mansfield Park of 4778 160460 0.02977689 0 0 8 Pride & Prejudice the 4331 122204 0.03544074 0 0 9 Emma of 4291 160996 0.02665284 0 0 10 Pride & Prejudice to 4162 122204 0.03405780 0 0 # ... with 40,369 more rows
A tibble: 27,267 x 2 ## item1 value ## <chr> <dbl> ## 1 windows 0.0357 ## 2 microsoft 0.0281 ## 3 ms 0.0245 ## 4 visual 0.0195 ## 5 linux 0.0188 ## 6 studio 0.0178 ## 7 net 0.0171 ## 8 desktop 0.0164 ## 9 xp 0.0163 ## 10 office 0.0147 ## # ... with 27,257 more rows