Share
Embed
Structure of NLP Papers
The slides for paper writing tutorial 2022 in NAIST NLP Lab.
Ukyo Honda
May 19, 2022
Transcript
36 slides
-
1 0WFSWJFX • 5JQTPOIPXUPTUSVDUVSFZPVSQBQFS • 4USVDUVSFPGTFDUJPOT • 4USVDUVSFJOTJEFPGFBDITFDUJPO • .BOZUIBOLTUPNZDPBVUIPST
NFOUPST BOEUIJTBXFTPNFUVUPSJBM • IUUQXXXQIPOUSPODPNTMJEFTOFVCJHOMQUVUPSJBMQEG -
2 $BWFBUT • 5IJTUVUPSJBMBTTVNFTUIBUZPVSQBQFSTBSF • *O/-1PSPUIFSDMPTFEPNBJOTTVDIBT$7BOE.- • /PUTVSWFZUIFNFEFNPQBQFST • 5IJTUVUPSJBMEPFTOPUDPWFSBMMTFDUJPOT
• 4FFUIFMJOLCFMPXGPSUIFOFXNBOEBUPSZTFDUJPOT • -JNJUBUJPOT &UIJDBM 3JTLT FUD • IUUQTBDMSPMMJOHSFWJFXPSHSFTQPOTJCMF/-1SFTFBSDI • 5IFUJQTBSFCBTFEPONZQFSTPOBMQSBDUJDFBOEQSFGFSFODF • $PNNFOUTBOETVHHFTUJPOTBSFXFMDPNF -
4 3PVHITLFUDI 8IBUEPQBQFSTMPPLMJLF Abstract Introduction Methods Related Work Experiments Conclusion
References !Consists of many different sections !Each section seems to have different roles and contents "What should we write in each section? "How to make sure the consistency? -
5 'JOFSHSBJOFETLFUDI 8IBUEPQBQFSTMPPLMJLF Methods Related Work Experiments References !Papers are
intentionally redundant for better understanding !Repeat of the basic arguments in different resolution "Just focus on structuring the basic arguments! 1. Background 2. Methods 3. Experiments Methods Experiments Background Methods Experiments Background Background Methods Experiments -
6 #BTJDBSHVNFOUT #BDLHSPVOE Related Work Experiments References Methods Experiments Background
Methods Experiments Background Background Methods Experiments Methods -
7 #BTJDBSHVNFOUT #BDLHSPVOE • *OUSPEVDFTUIFQSPCMFNTZPVBEESFTTJOUIJTQBQFS (PBMTBOETFUUJOHTPGUIFUBTL #SJFGTLFUDIPGQSFWJPVTBQQSPBDIFT •
%FUBJMFEEFTDSJQUJPOHPFTUP3FMBUFE8PSL 3FNBJOJOHQSPCMFNT UPBEESFTTJOUIJTQBQFS v Describe in this order -
8 #BTJDBSHVNFOUT .FUIPET Related Work Experiments References Methods Experiments Background
Methods Experiments Background Methods Background Methods Experiments -
9 #BTJDBSHVNFOUT .FUIPET • %FTDSJCFTIPXUPTPMWFUIFQSPCMFNT *OUVJUJWFFYQMBOBUJPO • )PXZPVSNFUIPETTPMWFUIFQSPCMFNT
%FUBJMTPGZPVSNFUIPET v Describe in this order -
10 #BTJDBSHVNFOUT &YQFSJNFOUT Related Work Experiments References Methods Experiments Background
Methods Experiments Background Methods Background Methods Experiments -
11 #BTJDBSHVNFOUT &YQFSJNFOUT • %FNPOTUSBUFT UIBUZPVSNFUIPETTPMWFUIFQSPCMFNT PSOPU q &YQFSJNFOUBMTFUUJOHT q
0WFSBMMDPNQBSJTPOXJUICBTFMJOFTBOEQSFWJPVTNPEFMT • 5PWFSJGZUIBUZPVSNFUIPETBDIJFWFUIFHPBM CFUUFS PSOPU q "CMBUJPOTUVEJFT • 5PTFFXIJDIQBSUPGZPVSNFUIPETDPOUSJCVUFT PSOPU q 'PDVTFEBOBMZTJT • 5PGVSUIFSJOWFTUJHBUFIPX ZPVSNFUIPETBDIJFWFUIFHPBMCFUUFS PSOPU JF XIFUIFSZPVSNFUIPETTPMWFUIFQSPCMFN ZPVQPJOUFEPVU q FUD -
12 4VNNBSZTPGBS • 1BQFSTSFQFBUFEMZFYQMBJO (PBMTBOEQSPCMFNT #BDLHSPVOE 4PMVUJPOT .FUIPET
%FNPOTUSBUJPO &YQFSJNFOUT • 5IJTSFQFBUFOTVSFT • $MFBSFYQMBOBUJPOPGZPVSBSHVNFOUT • $POTJTUFODZUISPVHIUIFFOUJSFQBQFS -
14 &YBNQMF • Hereafter we use this paper as the
example • https://arxiv.org/abs/2202.06417 -
15 &YBNQMF 0VUMJOF Abstract Introduction Methods Related Work Experiments Conclusion
References Background • Slightly different in the colored sections • But basically the same as the structure we have seen above -
17 'JOFSHSBJOFETLFUDI "CTUSBDU ! Explanation in order 1. Task and
goal • Text generation for app. 2. Prev. approaches • Stochastic sampling • Modified objective function 3. Remaining problems • Lack of coherence • Anisotropic dist. of rep. • Solutions • Calibrate the rep. space • Diversity-aware decoding • Demonstration • Outperform SOTA 1. 2. 3. -
18 &YBNQMF *OUSPEVDUJPO !5IFTBNFTUSVDUVSF BTUIF"CTUSBDU • #VUXJUINPSFEFUBJMT • *OIJHIFSSFTPMVUJPO •
Background • Methods • Experiments -
19 &YBNQMF #BDLHSPVOE • &YUFOEFE #BDLHSPVOE • &MBCPSBUJPOPOUIFUBTLTFUUJOHT • 4PNFUJNFT#BDLHSPVOEJTFYUFOEFEBOEDVUPVUGPSFMBCPSBUJPO
• 5PJOUSPEVDFGPSNBMUBTLTFUUJOHT • 5PFYQMBJOQSFWJPVTMZVOLOPXOQSPCMFNT • FUD Methods Related Work Experiments References Methods Experiments Background Methods Experiments Background Background Methods Experiments Background -
20 &YBNQMF .FUIPEPMPHZ ! Explanation in order 1. Intuitive explanation
on how your methods solve the problem 2. Then describe the details 1. 2. 1. 2. -
21 &YBNQMF &YQFSJNFOUT • %PDVNFOU(FOFSBUJPO • FYQFSJNFOUBMTFUUJOHT • 3FTVMUT •
)VNBO&WBMVBUJPO • 0QFO%PNBJO%JBMPHVF(FOFSBUJPO • FYQFSJNFOUBMTFUUJOHT • SFTVMUT • 'VSUIFS"OBMZTJT q Experimental settings q Overall comparison q Ablation study q Experimental settings q Overall comparison q Ablation study q Focused analysis -
22 0WFSBMMDPNQBSJTPO &YQFSJNFOUT q Overall comparison • Demonstrates that the
proposed method achieves the goal better than the SOTA methods -
23 q Ablation study • Two proposed methods • Each
of them works well and works the best in combination "CMBUJPOTUVEZ &YQFSJNFOUT -
24 'PDVTFEBOBMZTJT &YQFSJNFOUT q Focused analysis • Proposed methods improve
the overall performance, but how? ü Do they work as you hypothesized? ü Do they solve the problem you pointed out? Ø Confirming here that the methods mitigate the anisotropic distribution of token representations -
26 &YBNQMF $PODMVTJPO !4BNFBT"CTU*OUSP CVU • -FTTGPDVTPO#BDLHSPVOE • 1SPCMFNPOMZ •
.PSFGPDVTPO.FUIPETBOE&YQ • .PSFDPODJTFUIBO"CTU • +VTUUPTVNNBSJ[FSFFNQIBTJ[F ZPVSDPOUSJCVUJPOT • 'VUVSF8PSL -
27 &YBNQMF 3FGFSFODFT • 6TFBVOJGJFEGPSNBU ! *O"$- *O/""$-
" *O"$- *O1SPDFFEJOHTPGUIFUI"OOVBM.FFUJOHPGUIF"TTPDJBUJPOGPS $PNQVUBUJPOBM-JOHVJTUJDT *O1SPDFFEJOHTPGUIF$POGFSFODFPGUIF/PSUI"NFSJDBO$IBQUFS PGUIF"TTPDJBUJPOGPS$PNQVUBUJPOBM-JOHVJTUJDT • IUUQTZPTIJUBLBVTIJLVOFUIPX@UP@XSJUF@SFGFSFODFTIUNM -
28 &YBNQMF "QQFOEJY3FMBUFE8PSL • #SJFGEFTDSJQUJPOPGQSFWJPVTBQQSPBDIFTBOEEJGGFSFODF GSPNZPVST • 3FMBUFE8PSLPGUFODPNFTSJHIUBGUFS*OUSPEVDUJPO • 4PVOETGBJSBTJUJTBQBSUPG#BDLHSPVOE
• #VUQFSTPOBMMZ*EPOPUFNQMPZUIBUTUSVDUVSF • 1SFWJPVTBQQSPBDIFTBSFOPUZPVSNBJOQPJOUTDPOUSJCVUJPOT • 3FMBUFE8PSLUFOETUPCFUPPMPOH Ø+VTUHPPOUP.FUIPETGJSTUBOEHFUCBDLCFGPSF$PODMVTJPO v3FMBUFE8PSLJOBQQFOEJYIFSFJTBOFYUSFNFDBTFBOEOPUSFDPNNFOEFE -
30 5JQT 8SJUF"CTUSBDUGJSTU • "CTUSBDU≈ 0VUMJOF • %POUCFXPSSJFEJGUIJTTUFQUBLFTUPPMPOH • 3FNFNCFSUIBUQBQFSTDPOTJTUPG"CTUSBDUJOEJGGFSFOUSFTPMVUJPOT
• *GZPVBSFEPOFXJUIUIJT ZPVBSFBMNPTUUIFSF • 4IBSFZPVSESBGUPG"CTUSBDUXJUIZPVSDPBVUIPSTBOESFWJTF • 3FWJTFJUCBTFEPOGFFECBDLCFGPSFHPJOHPOUPUIFPUIFSTFDUJPOT • %POUIBWFUPCFTFOTJUJWFBCPVUMFOHUIBUUIFESBGUTUBHF -
31 5JQT 8SJUF"CTUSBDUCFGPSFTUBSUJOHSFTFBSDI • +VTUBTIPSUNFNPJTGJOF • *UIFMQTZPVUP 4NPPUIMZNPWFPOUPXSJUJOHQBQFST
,FFQJONJOEZPVSIZQPUIFTJTEVSJOHSFTFBSDI ≈ ,FFQJONJOEXIBUUPEPOFYU • 8IBUJTUIFHPBMBOEXIBUJTUIFSFNBJOJOHQSPCMFN #BDLHSPVOE • 8IBUJTUIFQPTTJCMFTPMVUJPO UPUIFQSPCMFN .FUIPET • )PXTIPVMEZPVEFNPOTUSBUF UIBUZPVSNFUIPETPMWFTUIFQSPCMFN &YQFSJNFOUT 3PVHIMZKVEHFXIFUIFSZPVSSFTFBSDIJTJOUFSFTUJOHPSOPUCFGPSFXBTUJOH UJNFPOJNQMFNFOUBUJPOFYQFSJNFOUT -
32 5JQT %POPUDPOGVTFFOETXJUINFBOT ! "DPNNPONJTUBLFJOGJOEJOHUIFQSPCMFNTUPTPMWF ! :PVSNFUIPESFTPVSDFFUDJTUIFNFBOT PGTPMWJOHQSPCMFNT • 5IFNFUIPESFTPVSDFFUDJUTFMGJTOPUUIFFOE
Ø/PUVTJOH9JTOPUBQSPCMFNUPTPMWF " 9IBTOPUCFFOVTFEJOUIFQSFWJPVTXPSL TPXFVTF9 FH $POUSBTUJWFMFBSOJOHIBTOPUCFFOVTFEJOUFYUHFOFSBUJPO TPXFVTFJU # 9DBOTPMWFUIJTQSPCMFNMFGUCZUIFQSFWJPVTXPSL TPXFVTF9 FH $POUSBTUJWFMFBSOJOHDBOTPMWFUIFBOJTPUSPQJDEJTUPGSFQ TPXFVTFJU -
33 8IZEPFTJUNBUUFS %POPUDPOGVTFFOETXJUINFBOT X Current Performance ! 9IBTOPUCFFOVTFEJOUIFQSFWJPVTXPSL TPXFVTF9 X
Better Performance Apply X ü Overall performance ü Ablation study ✘ Focused analysis • As you don't have any guess on how/why X works here • "X does something awesome!" • You don't have any hypotheses to test regarding how/why X works ! Shallow contribution -
34 8IZEPFTJUNBUUFS %POPUDPOGVTFFOETXJUINFBOT Problem X Current Performance ! 9DBOTPMWFUIJTQSPCMFNMFGUCZUIFQSFWJPVTXPSL TPXFVTF9
Problem X Better Performance Apply X ü Overall performance ü Ablation study ü Focused analysis • You can check how X solves the problem based on hypotheses • That gives the explanation of why X works here ! Deeper insight into the limitation/generalization of X ! More critical contribution -
35 5BLFIPNFNFTTBHFT! ü 1BQFSTSFQFBUFEMZFYQMBJO (PBMTBOEQSPCMFNT #BDLHSPVOE 4PMVUJPOT .FUIPET
%FNPOTUSBUJPO &YQFSJNFOUT ü 8SJUF"CTUSBDU JTI CFGPSFBOZUIJOH ü %POPUDPOGVTFFOETXJUINFBOT