Share
Embed
Transcript
21 slides
-
CompML TL; DR • Embedding ͷ֓ཁʹ͍ͭͯհ • Poincare Embeddings ͷ֓ཁʹ͍ͭͯհ
• ݪஶͱҟͳΔσʔληοτͰͷ࣮ݧ݁ՌΛհ 2 -
CompML Embedding ͱ (2/2) • ೖྗ: • ΦϒδΣΫτͷू߹: • ೋͭͷΦϒδΣΫτ
ͷڞىؔࢠؔΛࣔ͢σʔλ: • ग़ྗ: • ֤ΦϒδΣΫτͷ࠲ඪू߹: X = {xi |1 ≤ i ≤ N} xi , xj ∈ X D Y = {yi |1 ≤ i ≤ N} 5 -
CompML Embedding ͷྫ: Word2Vec (1/2) Skip-Gram [Mikolov et al., 2013[1]]:
֓ཁ • ฏۉରΛ࠷େԽ͢Δ͜ͱͰ จষ ͔Β ୯ޠϕΫτϧΛܭࢉՄೳ 1 T T ∑ t=1 ∑ −c≤j≤c,j≠0 logp(wt+j |wt ) p(wO |wI ) = exp(v′  T wO vwI ) ∑W w=1 exp(v′  T w vwI ) 6 -
CompML Embedding ͷྫ: Word2Vec (2/2) Skip-Gram [Mikolov et al., 2013[1]]:
࣮ݧ • ʮDNN ͷ embedding ʹར༻ͯ͠λεΫͷੑೳΛධՁʯͱ͍͏ྲྀΕͰͳ͍ • ఆੑධՁͷ໘ന͞ͱֶशΛΞϐʔϧ 7 -
CompML Poincare Embeddings ͱ (1/4) Poincare Space • ࠲ඪ ؒͷڑ͕ҎԼͰఆ·Δۂۭؒ
• த৺͔ΒΕΔ΄Ͳͷ͕૿͑Δٿঢ়ͷۭؒͰɺ ֊ੑͷ͋ΔΦϒδΣΫτΛ embed ͍͢͠ u, v d(u, v) = arcosh(1 + 2 ||u − v||2 (1 − ||u||2 )(1 − ||v||2 ) ) 9 -
CompML Poincare Embeddings ͱ (2/4) Embed ݁Ռͷྫ [Nickel et al.,
2017[2]] ಈͷ֊ੑΛ Wordnet ͔Βநग़ • Mammal -> Rodent • Rodent -> Squirrel 10 -
CompML Poincare Embeddings ͱ (3/4) ֶशํ๏ (WordNet ͷ߹) 1. ࢠؔͷ͋ΔΦϒδΣΫτೋͭͷͷू߹
Λੜ 2. ͔ΒωΨςΟϒαϯϓϧ Λੜ 3. ҎԼͷଛࣦؔΛ࠷దԽ֤ͯ͠ΦϒδΣΫτʹ࠲ඪΛ༩͑Δ D = {(u, v)|u ∈ v} D N( ⋅ ) 11 -
CompML Poincare Embeddings ͱ (4/4) ࣮ݧ: 3 छྨͷσʔληοτʹର͠ఆྔධՁ • Network
Reconstruction, Link Prediction (DNN ؔͳ͘ɺ Poincare Space ͰΦϒδΣΫτͷۙؔΛอ͍ͯͯΔ͔ධՁ) • ࣍ݩͰߴ͍ੑೳΛग़ͤΔ͜ͱΛΞϐʔϧ 12 -
CompML Poincare Embeddings ʹ͍ͭͯͷٙ ҎԼࡾͭͷσʔληοτҎ֎ʹ Citation Network Λ͑ͦ͏ • WordNet:
ࢠ͕ؔࣗ໌ͳ༗άϥϑ • ߏͷࢠؔΛͦͷ··ೖྗʹ༻Մೳ • Co-author NetWork: ࢠ͕ؔඇࣗ໌ͳແάϥϑ • ڞஶάϥϑ͔ΒΦϒδΣΫτؒͷۙ֬Λܭࢉ • Lexical Entailment: ΦϒδΣΫτؒͷؚҙؔΛࣔ͢ू߹ • X ͕ Y ʹଐ͢ΔఔΛ͔ࣔ͢ΒείΞ(ͱ Spearman’s ρ) Λܭࢉ 14 -
CompML Citation Network ͱ • ֓ཁ • จͷҾ༻ / ඃҾ༻ͷؔΛࣔͨ͠άϥϑ
• Ҿ༻ -> ࢠ, ඃҾ༻ -> ʹม͢Ε Poincare Embeddings Λద༻Ͱ͖ͦ͏ 15 -
CompML Citation Network Λ༻͍ͨ Poincare Embeddings (1/5) • ࣮ݧઃఆ •
ֶश • Ҿ༻ -> ࢠ, ඃҾ༻ -> ʹม͠ೖྗσʔλੜ • ೖྗσʔλʹର͠ Poincare Embeddings • ධՁ • ՄࢹԽʹΑΔఆੑධՁ • Network Reconstruction Error ʹΑΔఆྔධՁ • Ծઆ: WordNet ͱҟͳΓෳͷߏ͕ଘࡏ͢ΔͷͰධՁԼ͕Δ 16 -
CompML Citation Network Λ༻͍ͨ Poincare Embeddings (2/5) ఆੑධՁ dim =
2 ͷ Embed ݁ՌΛՄࢹԽ • σʔλ͕ଟ͍ͷͰີू͍ͯ͠Δ • ֊͝ͱʹ͍ͯ͠ͳ͍ 17 -
CompML Citation Network Λ༻͍ͨ Poincare Embeddings (3/5) ఆྔධՁ (1/2) MAP
(Mean Average Precision) (ߴ͍΄Ͳྑ͍) • dim = 2 ͱͦͷଞͰ͕ࠩ͋Δ • Network ͰͷੑೳΑΓѱ͍ 18 -
CompML Citation Network Λ༻͍ͨ Poincare Embeddings (4/5) ఆྔධՁ (2/2): Mean
Rank (͍΄Ͳྑ͍) • dim = 2 ͱͦͷଞͰ͕ࠩ͋Δ • WordNet ͰͷੑೳΑΓѱ͍ (Network Ͱͷ Mean Rank ݪஶະهࡌ) 19 -
CompML Citation Network Λ༻͍ͨ Poincare Embeddings (5/5) ࣮ݧͷ·ͱΊ • Citation
Network ΛφΠʔϒʹֶशͤͯ͞จ΄Ͳͷੑೳग़ͳ͔ͬͨ • ͱͯ͠Χϯτ͞Ε͍ͯΔΦϒδΣΫτͷݸΛੳͯ͠σʔλͷҧ͍ ΛௐΔඞཁ͋Γ • ࠓճ Citation Network ͱ gensim ͷ WordNet ·ͰੳࡁΈ • gensim ͱ Facebook Research Ͱσʔληοτ͕ҟͳΔͷͰɺ·ͩޙऀ ͷσʔλΛੳͰ͖ͯͳ͍ 20 -
CompML ࢀߟจݙ [1] Mikolov, Tomas, Sutskever, Ilya, Chen, Kai, Corrado,
Greg, and Dean, Jeffrey. Distributed representations of phrases and their compositionality. In Advances on Neural Information Processing Systems, 2013. [2] Maximillian Nickel and Douwe Kiela. Poincare embeddings for learning hierarchical representations. In Advances in Neural Information Processing Systems, 2017. 21