Share
Embed
Interpretable Representations for Affect and Semantics
While NLP has advanced considerably in recent years, state-of-the-art models can no longer easily be inspected and understood directly. This talk presents a series of interpretable approaches for affect and semantics-related settings.
The first part of the talk will present interpretable vectors custom-tailored for emotions and their connections to words, fonts, and colors. Subsequently, I will discuss NLP approaches for emojis and their connection to emotion. The final part of the talk will consider structured representations to better model the structure of a sentence or other kinds of knowledge that may be useful in downstream applications.
Gerard de Melo
July 01, 2021
Transcript
91 slides
-
Interpretable Representations for Affect and Semantics Gerard de Melo http://gerard.demelo.org
Interpretable Representations for Affect and Semantics Gerard de Melo http://gerard.demelo.org -
Machine Learning Machine Learning Classifier Model Model Examples Learning Learning
??? Negative Positive -
Machine Learning Machine Learning Probably Positive! Prediction Prediction Classifier Model
Model Examples Learning Learning Negative Positive -
Machine Learning Machine Learning Probably Positive! Prediction Prediction Classifier Model
Model Examples Learning Learning Negative Positive -
Probably Incorrect! Prediction Prediction Classifier Model Model Examples Learning Learning
Negative Positive Machine Learning Machine Learning Image: Method Matters Blog -
Machine Learning Nowadays Machine Learning Nowadays Probably Positive! Classifier Model
Model Examples Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network -
AI as a Black Box AI as a Black Box
Probably Positive! Prediction Prediction Classifier Model Model Labelled Examples Learning Learning Negative Positive -
Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable
AI (XAI)? Why Explainable AI (XAI)? https://www.idealrole.com/blog/cv-bias -
Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable
AI (XAI)? Why Explainable AI (XAI)? -
Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion
Vectors for Emojis Trees for Sentences Graphs for Knowledge -
Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion
Vectors for Emojis Trees for Sentences Graphs for Knowledge -
Modern AI Modern AI Probably Positive! Classifier Model Model Examples
Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network -
Modern AI Modern AI Probably Positive! Classifier Model Model Examples
Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network -
0.00 0.23 0.03 0.31 0.01 0.03 0.91 ... 0.31 0.50
Embeddings Embeddings 0.01 0.23 0.01 0.29 0.00 0.03 0.92 ... 0.30 0.51 “dry” “arid” ≈ x x x x amazing awesome unbearable horrible x terrible xstupendous Vector space in which similar words have similar vectors Vector space in which similar words have similar vectors -
Question Question Question Question Image: https://www.synthesio.com/blog/social-sentiment-and-your-business/ Can we have vector
representations that are interpretable and provide detailed information about the sentiment/emotion of a word? Can we have vector representations that are interpretable and provide detailed information about the sentiment/emotion of a word? 0.00 0.23 0.03 0.31 0.01 0.03 0.91 ... 0.31 0.50 0.01 0.23 0.01 0.29 0.00 0.03 0.92 ... 0.30 0.51 “stupendous” “amazing” ≈ ?? -
Multi-Dimensional Multi-Dimensional Sentiment Embeddings Sentiment Embeddings Multi-Dimensional Multi-Dimensional Sentiment Embeddings
Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 Music Laptops Image: http://www.thewilliamnyc.com/william-gallery/ 0.97 -0.30 ... 0.90 0.94 ... “hot” “amazing” -
Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven
Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 -
Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven
Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 f Feature Value “horrible” 1 “great” 2 “inside” 1 “cat” 3 ... ... -
Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven
Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 f Feature Value Weight “horrible” 1 -5.2 “great” 2 4.4 “inside” 1 -0.1 “cat” 3 1.2 ... ... ... -
Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven
Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 Train n linear models Used 25 different domains from Amazon Books, Electronics, Movies, Kitchen, etc. Used 25 different domains from Amazon Books, Electronics, Movies, Kitchen, etc. -
Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven
Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 0.0 0.8 0.2 ... –0.8 0.1 hot Train n linear models For each word, turn its linear coefficients across different n models into a single vector ... -
Emotion Analysis Emotion Analysis Emotion Analysis Emotion Analysis “What the
hell is going on?! I've been waiting for 2 weeks now!!” “What the hell is going on?! I've been waiting for 2 weeks now!!” -
Emotion Models Emotion Models Emotion Models Emotion Models Image: Paul
Ekman Ekman's 6 Basic Emotion Ekman's 6 Basic Emotion -
Emotion Models Emotion Models Emotion Models Emotion Models Image: https://commons.wikimedia.org/wiki/File:Plutchik-wheel.svg
Plutchik's Wheel of Emotions Plutchik's Wheel of Emotions -
Even More Emotions Even More Emotions Even More Emotions Even
More Emotions Cowen & Keltner (2017) Are 8 basic emotions really sufficient? Are 8 basic emotions really sufficient? -
AffectVec AffectVec AffectVec AffectVec https://www.flickr.com/photos/ill-padrino/6437837857/ joy sadness anger guilt suspense
0.5 0.1 0.4 ... 0.3 0.1 prank Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database Matej Kren: Idiom. Prague Municipal Library Prank Image: CollegeHumor -
AffectVec AffectVec AffectVec AffectVec Shahab Raji, Gerard de Melo. What
Sparks Joy: The AffectVec Emotion Database -
AffectVec AffectVec AffectVec AffectVec Shahab Raji, Gerard de Melo. What
Sparks Joy: The AffectVec Emotion Database -
AffectVec AffectVec AffectVec AffectVec Download: http://emotionlexicon.org/ Download: http://emotionlexicon.org/ Shahab Raji,
Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database Also: Experiments on Unsupervised Document-Level Emotion Prediction -
Cross-Lingual Cross-Lingual Emotion Lexicon Induction Emotion Lexicon Induction Cross-Lingual Cross-Lingual
Emotion Lexicon Induction Emotion Lexicon Induction Over 300 languages http://emotionlexicon.org/ Over 300 languages http://emotionlexicon.org/ Arun Ramachandran, Gerard de Melo. COLING 2020 -
Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu,
Gerard de Melo. Predicting Semantic Signatures of Fonts Van Rompay and Pruyn (2011) Choice of font affects assumed brand credibility price expectations Choice of font affects assumed brand credibility price expectations -
Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu,
Gerard de Melo. Predicting Semantic Signatures of Fonts de Sousa et al. (2020). Journal of Sensory Studies Choice of font affects intent to purchase and assumed acidity, sweetness Choice of font affects intent to purchase and assumed acidity, sweetness -
Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu,
Gerard de Melo. Predicting Semantic Signatures of Fonts Shaikh (2007) O’Brien Louch & Stork (2014) Choice of font affects assumed attributes of people using them Choice of font affects assumed attributes of people using them -
Font Selection Font Selection Font Selection Font Selection Tugba Kulahcioglu,
Gerard de Melo. Predicting Semantic Signatures of Fonts -
Learning Representations of Fonts Learning Representations of Fonts Learning Representations
of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts -
Learning Representations of Fonts Learning Representations of Fonts Learning Representations
of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts -
Learning Representations of Fonts Learning Representations of Fonts Learning Representations
of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts fontjoy.com -
Learning Representations of Fonts Learning Representations of Fonts Learning Representations
of Fonts Learning Representations of Fonts http://fontjoy.com/projector/ Visual Representation Space Visual Representation Space -
Learning Representations of Fonts Learning Representations of Fonts Learning Representations
of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts fontjoy.com Infer Attribute-Based Representations Infer Attribute-Based Representations -
Learning Representations of Fonts Learning Representations of Fonts Learning Representations
of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts -
FontLex FontLex FontLex FontLex Tugba Kulahcioglu, Gerard de Melo. FontLex:
A Typographical Lexicon based on Affective Associations Infer font–emotion mapping using seed data + word2vec Infer font–emotion mapping using seed data + word2vec -
FontLex FontLex FontLex FontLex via Sentiment/Emotions associations of words Tugba
Kulahcioglu, Gerard de Melo. FontLex: A Typographical Lexicon based on Affective Associations -
FontLex: FontLex: Complex Emotions Complex Emotions FontLex: FontLex: Complex Emotions
Complex Emotions Tugba Kulahcioglu, Gerard de Melo. Semantics-Aware Typographical Choices via Affective Associations Extension by using more fonts for FontLex data -
Emotions vs. Emotions vs. Font and Color Font and Color
Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI Fonts: best for serious, trustworthy, disturbing Color palettes: best for calm, negative, playful -
Emotions vs. Emotions vs. Font and Color Font and Color
Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI United Nations -
Emotions vs. Emotions vs. Font and Color Font and Color
Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. FontLex: A Typographical Lexicon based on Affective Associations / Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI The Smurfs Scream -
Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion
Vectors for Emojis Trees for Sentences Graphs for Knowledge -
Emojis are ubiquitous Emojis are ubiquitous Emojis are ubiquitous Emojis
are ubiquitous Image: https://www.theverge.com/2016/2/24/11105250/facebook-reactions-emoji-how-to Very prominent on social media, instant messaging email subject lines etc. Very prominent on social media, instant messaging email subject lines etc. Face with Tears of Joy: Oxford Dictionaries Word of the Year 2015 Face with Tears of Joy: Oxford Dictionaries Word of the Year 2015 -
Emojis vs. Emotions Emojis vs. Emotions Emojis vs. Emotions Emojis
vs. Emotions Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Same parts of brain activated as when seeing a real face Same parts of brain activated as when seeing a real face “Emoji” is Japanese for “picture character” but clearly closely linked to emotions Image: from Petra Kralj Novak. Sentiment of Emojis -
Emoji-Based Emoji-Based Word Embeddings Word Embeddings Emoji-Based Emoji-Based Word Embeddings
Word Embeddings Image: https://www.theverge.com/2016/2/24/11105250/facebook-reactions-emoji-how-to 0.8 0.4 0.0 ... 0.0 0.1 prank Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP -
Use Case: Deep Neural Network for Use Case: Deep Neural
Network for Tweet Emotion Classification Tweet Emotion Classification Use Case: Deep Neural Network for Use Case: Deep Neural Network for Tweet Emotion Classification Tweet Emotion Classification Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Methods Anger Fear Joy Sadness Average Dim Interpretable Affective Tweets 0.65 0.66 0.60 0.69 0.65 n/a EmoTag 0.70 0.73 0.69 0.75 0.72 620 Non-Interpretable Random Int. 0.68 0.72 0.66 0.73 0.70 300 word2vec 0.70 0.72 0.67 0.75 0.71 300 GloVe 0.70 0.73 0.68 0.76 0.72 300 GloVe Twitter 0.72 0.74 0.68 0.76 0.73 200 -
Use Case: Emoji Sentiment Use Case: Emoji Sentiment Use Case:
Emoji Sentiment Use Case: Emoji Sentiment Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP -
Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions
Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Abu Shoeb, Gerard de Melo. EmoTag1200 : Understanding the Association between Emojis and Emotions . EMNLP 👍 😄 😻 Abu Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Online: emoji.nlproc.org Online: emoji.nlproc.org -
Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions
Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Abu Shoeb, Gerard de Melo. EmoTag1200 : Understanding the Association between Emojis and Emotions . EMNLP 👍 😄 😻 Abu Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP -
Emojis in NLP: Emojis in NLP: Sentiment Analysis Sentiment Analysis
Emojis in NLP: Emojis in NLP: Sentiment Analysis Sentiment Analysis Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021 -
Emojis in NLP: Emojis in NLP: Tokenization Tokenization Emojis in
NLP: Emojis in NLP: Tokenization Tokenization Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021 -
Complex Emojis Complex Emojis Complex Emojis Complex Emojis Abu Shoeb,
Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021 -
Emojis in NLP: Emojis in NLP: POS Tagging POS Tagging
Emojis in NLP: Emojis in NLP: POS Tagging POS Tagging Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021 -
Emojis in NLP: Emojis in NLP: Dependency Parsing Dependency Parsing
Emojis in NLP: Emojis in NLP: Dependency Parsing Dependency Parsing Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021 -
Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion
Vectors for Emojis Trees for Sentences Graphs for Knowledge -
What are they capturing? What are they capturing? What are
they capturing? What are they capturing? Image: Adapted from http://www.cse.unt.edu/people/StudentNewsletters/2013_Apr_StudentEmailNewsletter.html you can't cram the meaning of a whole ***ing sentence into a single ***ing vector -
Evaluation via Evaluation via Probing Probing Evaluation via Evaluation via
Probing Probing Also: Adi et al. ICLR 2016 These test whether enough information is kept to learn something from 100,000 training examples. These test whether enough information is kept to learn something from 100,000 training examples. -
Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure
Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure A person is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* sim(S0,S=) sim(S0,S=) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018 -
Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure
Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure A person is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* sim(S0,S=) sim(S0,S=) sim(S0,S*) sim(S0,S*) sim(S0,S=) > sim(S0,S*) ? -
Negation Detection Negation Detection Negation Detection Negation Detection A person
is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018 -
Negation Variant Negation Variant Negation Variant Negation Variant A man
is not standing on his head under water. There is no man standing on his head under water. A man is standing on his head under water. S0 S= S* (Negation) (Negated Existential) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018 -
Clause Relatedness Clause Relatedness Clause Relatedness Clause Relatedness Octel said
the purchase was expected. The purchase was expected. Octel said the purchase was not expected. S0 S= S* Clause Extraction (for suitable head verbs only) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018 -
Argument Sensitivity Argument Sensitivity Argument Sensitivity Argument Sensitivity Francesca teaches
Adam to adjust the microphone on his stage. Adam is taught to adjust the microphone on his stage. Adam teaches Francesca to adjust the microphone on his stage. S0 S= S* (Passive) (Argument Inversion) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018 -
Argument Sensitivity Argument Sensitivity Argument Sensitivity Argument Sensitivity Zhu, Li,
de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018 Who-did-what- to-whom not reflected in topology Who-did-what- to-whom not reflected in topology -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling black cat black cat the black the black cat the Which composition makes more sense? Which composition makes more sense? -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away black cat black cat the black the black cat the -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away black cat black cat the black the black cat the -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Local Probability Compositional Embedding For each split point k of T1,3 -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Local Probability Compositional Embedding For each split point k of T1,3 Weighted sum over all split points -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Unsupervised Language Model Pre-Training Unsupervised Language Model Pre-Training -
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model
R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling -
Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion
Vectors for Emojis Trees for Sentences Graphs for Knowledge -
Universal Wordnet Universal Wordnet Universal Wordnet Universal Wordnet cmn: “
” 机构 http://lexvo.org/uwn/ Gerard de Melo. Towards a Universal Wordnet by Learning from Combined Evidence. -
Explainable Inference in Graphs Explainable Inference in Graphs Fu et
al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html -
Explainable Inference in Graphs Explainable Inference in Graphs Fu et
al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html -
Explainable Inference in Graphs Explainable Inference in Graphs Fu et
al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html In reality, there are many possible paths. Which to choose? In reality, there are many possible paths. Which to choose? -
Explainable Inference in Graphs Explainable Inference in Graphs Fu et
al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html Reward if you guessed correctly! Reward if you guessed correctly! -
Explainable Inference in Graphs Explainable Inference in Graphs Xian et
al. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation. SIGIR 2019 -
x x 감사합니다 gracias xthank you Summary Summary Summary Summary
Vectors for Sentiment and Emotion ►Interpretable Vectors ►Font Affect Embeddings Emojis ►Interpretable Vectors ►NLP Analysis Trees for Sentences ►Analysis ►R2D2 Differentiable Tree Model Graphs for Knowledge ►Reinforcement Learning over Knowledge Graphs for Explainable AI Get in Touch! http://gerard.demelo.org [email protected] Get in Touch! http://gerard.demelo.org [email protected] x 谢谢 x Vielen Dank спаси́бо x x obrigado