Same content. Different words.

Introducing Word Mover's Distance in gensim.

Transcript

1. Same content. Different words. Lev Konstantinovskiy Community Manager at Gensim @teagermylk http://rare-technologies.com/
2. Streaming We turn NLP papers into industrial Python code.
3. Credits Olavur Mortensen Applied Mathematics student DTU in Copenhagen RaReTech Incubator program Added WMD to Gensim http://rare-technologies.com/incubator/
4. Business Problem All reviewers are raving about the same thing “The Sicilian gelato was extremely rich” “The Italian ice-cream was very velvety” What about Ambiance, Service and Prices? Let’s filter “gelato” out and add other aspects! Credit: Sudeep Das @datamusing applied WMD to restaurant reviews. http://tech.opentable. com/2015/08/11/navigating-themes-in-restaurant-reviews-with-word-movers-distance/
5. Ways to find similar documents • Count common words ( bag of words, TF-IDF) ◦ #Dimensions = #Vocabulary (thousands) Stuck if no words in common. “Gelato” != “Ice-cream”
6. Ways to find similar documents • Low-dimensional latent features ◦ Eigen-values (LSI) ◦ Probability (LDA) Nice features. That is basically what most of Gensim is. But there is something better now… WMD!
7. New way to find similar documents • Word Mover’s Distance ◦ Built on top of Google’s word2vec ◦ Well-used concept in other fields known as Earth Mover’s Distance Beats BOW, TF-IDF, LDA, LSI in k Nearest Neigbours classification tasks.
8. Word Mover’s distance http://jmlr.org/proceedings/papers/v37/kusnerb15.pdf https://github.com/mkusner/wmd
9. Word Mover’s distance http://tech.opentable.com/2015/08/11/navigating-themes-in-restaurant-reviews-with-word-movers-distance/
10. Distance in gensim
11. Finding similar reviews from gensim.similarities import WmdSimilarity similiar_reviews = WmdSimilarity(reviews, model, num_best=10) similar_reviews['Very good, you should seat outdoor.']
12. Thanks! Lev Konstantinovskiy github.com/tmylk @teagermylk Events coming up: - Word2vec successor Swivel DS Journal Club Meetup 21 April - Word embeddings talk at PyData London conference 7-8 May - Word2vec tutorial at PyData Berlin 20-21 May
13. Extra slides
14. Ways to find similar documents • Google’s Doc2vec ◦ Built on top of word2vec ◦ Document tags are just extra words in the document Hard to tune. Slow inference.
15. Earth Mover’s Distance How do you best move piles of sand to fill up holes of the same total volume? Stated by Monge in 1781. Solved by Kantorovich in [Image: APS/Alan Stonebraker]
16. Google’s Word2vec algorithm • Word becomes a vector in 100-dimensional space. • king - man + woman = queen http://nbviewer.jupyter.org/github/fbkarsdorp/doc2vec/blob/master/doc2vec.ipynb http://radimrehurek.com/2014/02/word2vec-tutorial
17. http://vene.ro/blog/word-movers-distance-in-python.html Word Mover’s distance