Share
Embed
Transcript
18 slides
-
Introduction • 模型建立在latent tree chart parser的现有工作上。 • - Semi-supervised recursive
autoencoders for predicting sentiment distributions • - The forest convolutional network: Compositional distributional semantics with a neural chart and without binarization • - Learning to compose task-specific tree structures, AAAI 2018 • - Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs • CKY算法也是一种chart parser。 -
DIORA的贡献 • 将Inside-Outside算法合并到latent tree chart parser中 • 不需要annotation,以language model为训练目标。 •
无监督方法中,DIORA是state-of-the-art,且树最深 -
Inside Pass • 对于任意的一个高层节点k,儿子节点对的组合为{k}。 • 这里k对应了一个span。K范围里的相邻节点对(i, j) ∈ {k} •
对每个节点对做两个运算: • (i, j)的相容分数e(i, j) • (i, j)的向量a(i, j) -
Inside Pass • 再看组合向量composition vector。 • 这个分数和这个vector表示了cell k的子树软加权。 • 有两种方法实现这个函数:Tree-LSTM和MLP。
Tree-LSTM中,inside时w=1,outside时w=0 MLP做的是concat -
Outside Pass • 对于下层节点k, • 计算父节点j和兄弟节点i的composition score, • 约定为f •
Compose函数inside和outside共享参数。 -
Generate a Tree • 有多种办法,这里使用传统的CKY 算法。 • CKY算法可以为CFG和PCFG以动态 规划的方式parse出一个树 •
所以这里只用inside分数e(i, j) -
Experiment • 模型在三个task上应用: • Unsupervised Parsing • Phrase Segmentation •
看模型识别出了句子中有多少个短语 • Phrase Similarity • 首先用DIORA算出每个短语的表示,然后两两短语之间算cos相似 度,对于一个短语,如果与它最相似的K个短语的label和它一样, 那么这个短语就预测对了。 -
相关论文 • Semi-supervised recursive autoencoders for predicting sentiment distributions •
The forest convolutional network: Compositional distributional semantics with a neural chart and without binarization • Learning to compose task-specific tree structures, AAAI 2018 • Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs • Do latent tree learning models identify meaningful structure in sentences? • Grammar induction with neural language models: An unusual replication • Structured alignment networks for matching sentences • graph-based dependency parser + beam search: The insideoutside recursive neural network model for dependency parsing • Neural crf parsing