Upgrade to Pro — share decks privately, control downloads, hide ads and more …

文献紹介: Complexity-Weighted Loss and Diverse Rera...

文献紹介: Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification

2019/04/23の文献紹介で発表

Yumeto Inaoka

April 23, 2019
Tweet

More Decks by Yumeto Inaoka

Other Decks in Research

Transcript

  1. Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification 文献紹介 (2019/04/23)

    長岡技術科学大学 自然言語処理研究室 稲岡 夢人
  2. Literature Title Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification

    Author Reno Kriz, João Sedoc, Marianna Apidianaki, Carolina Zheng, Gaurav Kumar, Eleni Miltsakaki, Chris Callison-Burch Conference NAACL-HLT 2019 Paper URL https://arxiv.org/abs/1904.02767 2
  3. •Complexity-Weighted Loss Function - Word Complexity Prediction - Loss Function

    •Reranking Diverse Candidates •Diverse Candidate Simplifications 8
  4. Reranking Diverse Candidates •𝑠𝑠𝑖𝑖 = 𝑖𝑖 + 𝑖𝑖 + 𝑖𝑖

    •𝑖𝑖 , 𝑖𝑖 , 𝑖𝑖 : 流暢さ、意味保持性、平易さ •𝑖𝑖 , , : 各要素の重み 14
  5. Diverse Candidate Simplifications • −1 , ,′ = log 1

    , … , −1 , ,′ − ′ ∗ •−1 : 時間 − 1における位の出力 • ,′ : −1 から出力される位のトークン •, ′ ∈ 1. . •はハイパーパラメータ 16
  6. Diverse Candidate Simplifications Jiwei Li, Will Monroe, and Dan Jurafsky.

    A Simple, Fast Diverse Decoding Algorithm for Neural Generation. 2016. CoRR, abs/1611.08562. 17
  7. Training Details •Sockeye (Seq2seq framework built on MXNet) •損失関数における重み =

    2 •ビームサイズ = 100 •ビームサーチ時のペナルティ = 1.0 •候補選択時のクラスタサイズ = 20 •リランキング時の重み = = = 1 3 と = = 1 2 , = 0 20
  8. Evaluation (Human Eval.) •文長が人手評価に与える影響を調査 •Original : S2S-All-FAの結果 •MATCH-Dress0 : DRESS-Lenに近い文を選択

    •MATCH-Dress+2 : DRESS-Len +2 に近い文を選択 •MATCH-Dress-2 : DRESS-Len – 2 に近い文を選択 30
  9. Examples of Error Type •複数の句を含む長く複雑な文 Complex : Turkey has long

    enshrined the secular ideals of founding father Mustafa Kemal Ataturk, particularly in an education system that until recently banned Islamic headscarves in schools and made schoolchildren begin the day reciting an oath of allegiance to Ataturk’s legacy. Reference : Schools in Turkey had banned headscarves. Simple : They made schoolchildren to Ataturk’s history. 32
  10. Examples of Error Type •訓練コーパスのアライメントミスによる訓練の混乱 Complex : He heard the

    applause. Reference : The winning word was “magician.” Simple : But he heard the song. 33
  11. Examples of Error Type •照応解析が必要なもの Complex : He is the

    creative director of Rethink Leisure & Entertainment, which is working on several projects in China and elsewhere in Asia. Reference : He is with Rethink Leisure & Entertainment. Simple : He is working on several projects in China. 34
  12. Examples of Error Type •文の誤った部分を平易化 Complex : His father owned

    the home when the lava flowed slowly to the coast. Reference : His father still owned the home. Simple : The river cut slowly to the coast. 35
  13. Examples of Error Type •単語埋め込みの近さによる悪い換言 Complex : In Beijing kite

    circles, Fei is widely known as the elder statesman. Reference : In Beijing, Fei is widely known as an expert. Simple : In Beijing, Fei is considered a doctor. 36
  14. Examples of Error Type •訓練にない許容できる平易化 Complex : He later tried

    to buy his freedom. Reference : Henson tried to buy his freedom later. Simple : The man tried to buy his freedom. 37
  15. 40