Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
文献紹介: A Document Descriptor using Covariance of...
Search
Yumeto Inaoka
February 27, 2019
Research
340
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
文献紹介: A Document Descriptor using Covariance of Word Vectors
2019/02/27の文献紹介で発表
Yumeto Inaoka
February 27, 2019
Transcript
22 slides
A Document Descriptor using Covariance of Word Vectors 文献紹介 2019/02/27
長岡技術科学大学 自然言語処理研究室 稲岡 夢人
Literature 2 Title A Document Descriptor using Covariance of Word
Vectors Author Marwan Torki Volume Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 527-532, 2018.
Abstract 単語ベクトルを用いた固定長の文書表現を提案 (Document-Covariance Descriptor; DoCoV) → Supervised, Unsupervisedのアプリケーションで 簡単に利用できる
様々なタスクでSoTAに匹敵する性能 3
Introduction ベクトルを利用した文書検索には長い歴史がある ← Bag-of-Words, Latent Semantic Indexing(LSI) 近年はニューラル言語モデルで単語埋め込みを学習
単語ではなく文, 段落, 文書の分散表現も注目されている 4
vs. DoCoV doc2vecやFastSentは 単語と共通の空間 共分散は単語の密度の 形状を符号化 5
vs. DoCoV doc2vecやFastSentは学習に時間がかかる DoCoV(共分散)の計算は並列性が高く高速に行える 6
DoCoV Document Observation Matrix d次元の単語埋め込みとn単語の文書において ∈ ×と定義 (行は単語、列は埋め込みの各次元) 7
DoCoV Covariance Matrix 8
DoCoV Vectorized representation 9
Evaluation IMDB movie reviewsの分類性能によって 単語ベクトルによる変化を評価 ベクトルを線形SVMで分類 1つのレビューは複数の文で構成される
Train/Test/Unlabeled : 25K/25K/50K 事前学習済みのword2vec, GloVeと、 TrainとUnlabeledで学習したword2vecで比較 10
Result 11
Result 12
Result 13
Result 14
Evaluation 文の意味関連性データセットSICK, STS 2014で 文書ベクトルを評価 事前学習済みの単語埋め込みを使用 (dim=300)
Pearson correlationとSpearman correlationで評価 15
Result 学習が必要な他手法と匹敵するような結果 16
Evaluation Google newsで事前学習済みの単語埋め込みを使用 Movie Reviews(MR), Subjectivity(Subj), Customer Reviews(CR),
TREC Question(TREC)を データセットとして使用 17
Result 18
Result 19
Result 20
Result 21
Conclusions 文、段落、文書の新たなベクトル表現方法を提案 他手法のような反復の学習を必要としない Supervised, Unsupervisedのタスクにおいて その有用性を確認 22
More Decks by Yumeto Inaoka
See All by Yumeto Inaoka
文献紹介: Quantity doesn’t buy quality syntax with neural language models
yumeto
1
230
文献紹介: Open Domain Web Keyphrase Extraction Beyond Language Modeling
yumeto
0
300
文献紹介: Self-Supervised_Neural_Machine_Translation
yumeto
0
200
文献紹介: Comparing and Developing Tools to Measure the Readability of Domain-Specific Texts
yumeto
0
220
文献紹介: PAWS: Paraphrase Adversaries from Word Scrambling
yumeto
0
250
文献紹介: Beyond BLEU: Training Neural Machine Translation with Semantic Similarity
yumeto
0
340
文献紹介: EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing
yumeto
0
440
文献紹介: Decomposable Neural Paraphrase Generation
yumeto
0
260
文献紹介: Analyzing the Limitations of Cross-lingual Word Embedding Mappings
yumeto
0
310
Other Decks in Research
See All in Research
ThermoGS (ECCV2026)解説 — 4D Gaussian Splatting × Thermodynamics CVと物理の統合モデルとして
godel
PRO
0
120
[WebDB2026]セレンディピティ指向推薦システム再考 ―セレンディピティの原義・類型・発生過程に基づく設計指針―
recsyslab
PRO
0
230
La longévité des troupeaux laitiers: état des lieux et perspectives
institutdelelevage
PRO
0
150
ros2-perf-multihost: Automated Coordination Framework for Objective Architecture Evaluation in Distributed Systems
takasehideki
0
180
[最先端NLP勉強会2026] Agentic Rubrics as Contextual Verifiers for SWE Agents
rfujii
1
420
フィジカルAIの社会実装に向けて -AIエージェントからAgentic Roboticsへ-
hf149
1
2.8k
[最先端NLP勉強会2026] Checklists Are Better Than Reward Models For Aligning Language Models
nzw0301
1
350
The Conductor Pattern -Multi-Granularity Feedback for Creative Agents-
shibuiwilliam
2
120
AI時代に学術情報をどう探し、どう使うか: 研究の様々な場面と一研究者の経験から
masao
0
210
生成AIなんでも展示会vol6 LT登壇資料 NexteraBERT
rikkabotan7
0
220
Byzantine Eventual Consistency for Reliable Local-First Software
ept
0
410
Vers un système bovin allaitant multiperformant :leviers actionnés et trajectoire de réduction de l'empreinte carbone à travers 4 années d'expérimentation à FERM'INOV
institutdelelevage
PRO
0
150
Featured
See All Featured
Code Reviewing Like a Champion
maltzj
529
40k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
508
140k
SEOcharity - Dark patterns in SEO and UX: How to avoid them and build a more ethical web
sarafernandez
0
330
Unlocking the hidden potential of vector embeddings in international SEO
frankvandijk
0
1k
Practical Orchestrator
shlominoach
192
12k
Being A Developer After 40
akosma
PRO
92
590k
We Analyzed 250 Million AI Search Results: Here's What I Found
joshbly
2
2.1k
The SEO Collaboration Effect
kristinabergwall1
1
620
Impact Scores and Hybrid Strategies: The future of link building
tamaranovitovic
0
490
Utilizing Notion as your number one productivity tool
mfonobong
4
650
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
inesmontani
PRO
0
810
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
16
2.2k