Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
text_mining_slides_20180512
Search
Sponsored
·
SiteGround - Reliable hosting with speed, security, and support you can count on.
→
Leo Lu
May 12, 2018
Technology
110
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
text_mining_slides_20180512
Leo Lu
May 12, 2018
Transcript
29 slides
Text Mining and Data Viz 2018-05-12 leoluyi@iii Slides http://pcse.pw/6WHWJ ©
leoluyi, 2018 1
橕ෝ౯ 4 㸎瓽 Leo Lu 4 ݣय़ૡᓕ 4 ፓ獮ෝᰂᣟ禂๐率 4
Build data products 4 ETL 4 Models 4 Text mining 4 Viz 4 ... © leoluyi, 2018 2
Text Minning 窕纷 膏 ૡٍ㮉 © leoluyi, 2018 3
膑碻դጱૡٍ vs. 碝Ӯդጱૡٍ © leoluyi, 2018 4
犥獮౯㮉᮷አक़㾴Ո䌃ጱ䩚ᥜ tm + tmcn Rwordseg © leoluyi, 2018 5
֕ฎ蝡犚ॺկஃஃࣁӾ 䨝磪๚Ꭳጱ襊 © leoluyi, 2018 6
犡ॠ౯㮉ᥝአӞ犚碝ጱૡٍ © leoluyi, 2018 7
窕纷 Get data ➜ Tokenize ➜ Embedding ➜ Viz ➜
Model © leoluyi, 2018 8
Get data Get data ➜ Tokenize ➜ Embedding ➜ Viz
➜ Model 9
PTT ฎ疌疌ጱঅ๏ Get data ➜ Tokenize ➜ Embedding ➜ Viz
➜ Model 10
ྯॠ᮷磪盄ग़盄ग़ጱ䔂承碘 © leoluyi, 2018 11
ᛔ૩ጱ粖恝ᛔ૩䌃 devtools::install_packages( "leoluyi/PTTr") © leoluyi, 2018 12
Cleaning and preprocessing text ኸӥ虻懱牧݄ധ褾懱 © leoluyi, 2018 13
Tokenize Transform whole text into parts Get data ➜ Tokenize
➜ Embedding ➜ Viz ➜ Model 14
For English 4 normalization 4 stemming (扃䓄玲) 4 lemmatization (扃ࣳ螭ܻ)
4 POS tagging 4 ... Get data ➜ Tokenize ➜ Embedding ➜ Viz ➜ Model 15
Ӿ犲Ԓ穉斃墋㻌 4 䥁扃 4 犋䥁扃 4 POS tagging 4 ...
Get data ➜ Tokenize ➜ Embedding ➜ Viz ➜ Model 16
Semantic Parsing vs. Bag-of-Words © leoluyi, 2018 17
R tools 4 stringr 4 jiebaR Get data ➜ Tokenize
➜ Embedding ➜ Viz ➜ Model 18
Embedding (Encode, Feature Extraction) Get data ➜ Tokenize ➜ Embedding
➜ Viz ➜ Model 19
Embedding In a nutshell, Word Embedding turns text into numbers.
4 Embedding Layer1 4 Word2Vec 4 GloVe 4 doc2vec 4 sense2vec 1 https://machinelearningmastery.com/what-are-word-embeddings/ Get data ➜ Tokenize ➜ Embedding ➜ Viz ➜ Model 20
© leoluyi, 2018 21
Demo Information Retrieval Get data ➜ Tokenize ➜ Embedding ➜
Viz ➜ Model 22
Visualize 4 Dimension Reduction 4 t-sne 4 PCA 4 Clustering
4 Interactive or static plots Get data ➜ Tokenize ➜ Embedding ➜ Viz ➜ Model 23
Visualize 4 tsne::tsne() 4 prcomp() Get data ➜ Tokenize ➜
Embedding ➜ Viz ➜ Model 24
Model Get data ➜ Tokenize ➜ Embedding ➜ Viz ➜
Model 25
Tasks 4 Classification 4 獤觊 4 Clustering 4 ತ疨ፘ犲 4
Generative models 4 ᛔ㵕ኞ౮ Get data ➜ Tokenize ➜ Embedding ➜ Viz ➜ Model 26
አک磧盅᮷䨝మᥝ䌃ᛔ૩ጱ toolkit 4 Sparse Matrix manipulation 4 Informaiton retrieval tools
4 ... © leoluyi, 2018 27
Summary 1. Problem definition & specific goal: Get Curious About
Text 2. Finding Your Data 3. Preprocessing Your Data 4 Removing stopwords, Stemming, Segmentation, ... 4. Feature Extraction 4 Document-Term Matrix: tm, text2vec 4 Named Entity Recognition, POS tagging 4 Word embeddings: word2vec, GloVe 5. More Text Mining Skills 4 sentiment analysis 4 topicmodels, LDAViz: LDA 6. More Than Words - Visualizing Your Results © leoluyi, 2018 28
碍硁ᑀ䋊 㸎瓽 leoluyi@github https://leoluyi.github.io © leoluyi, 2018 29
More Decks by Leo Lu
See All by Leo Lu
R from Data Analysis to Production
leoluyi
1
180
2018-07-28_viz_talk
leoluyi
0
110
Other Decks in Technology
See All in Technology
そのイメージ、どのCIが作った? Artifact Attestationsで来歴を固定する / Which CI system created that image? Pinning provenance with Artifact Attestations
chmikata
0
230
Amazon Bedrock AgentCore の基礎とセキュリティ・ガバナンス
ren8k
0
240
【Oracle AI SPOTLIGHT ウェビナー】AI時代、狙われるデータと止められない業務をどう守るか?
oracle4engineer
PRO
1
110
20261113_AWS Organizations運用を下支えするCFn StackSetsの柔軟性
dkduppf
0
110
市販IoT機器や自作IoT機器とHomeAssistantを駆使して家の電気代をできる限り見える化する方法の考案
harutiro
0
410
Agent Routerで作るマルチテナント 推論基盤 #CNDW2026 / Building Multi-Tenant Inference Platform with Agent Router #CNDW2026
everpeace
5
580
Kansai WordPress Meetup@大阪『ブロックテーマを作る上で知っておくべきこと
torounit
0
260
3文字略語に騙されない! 世界一わかりやすい『EDR』超入門
kamadamakoto
1
330
Is the Website Dying?
miriamschwab
0
110
エンタープライズIAMについて - ID・権限をどう統制するか
oidfj
1
290
Kubernetes, Agones, WebRTC, GPU で作る クラウドゲーミング基盤を本番運用に乗せるまで
vivion
0
250
目を覚ませ!僕らの認証情報が何者かに侵略されてるぞ
tkito
2
480
Featured
See All Featured
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
2k
Between Models and Reality
mayunak
4
600
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
livdayseo
1
250
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
akashhashmi
0
550
世界の人気アプリ100個を分析して見えたペイウォール設計の心得
akihiro_kokubo
PRO
74
43k
Testing 201, or: Great Expectations
jmmastey
46
8.4k
Keith and Marios Guide to Fast Websites
keithpitt
413
23k
The Hidden Cost of Media on the Web [PixelPalooza 2025]
tammyeverts
2
580
SERP Conf. Vienna - Web Accessibility: Optimizing for Inclusivity and SEO
sarafernandez
2
1.7k
Introduction to Domain-Driven Design and Collaborative software design
baasie
1
1.1k
The #1 spot is gone: here's how to win anyway
tamaranovitovic
4
1.2k
Public Speaking Without Barfing On Your Shoes - THAT 2023
reverentgeek
1
670