Introduction to LSTM

-45.ೖ໳ Kenji Hayashida

-45.ͱ͸ MPOHTIPSUUFSNNFNPSZ SFDVSSFOUOFVSBMOFUXPSLNPEFMͷҰछ γʔέϯεσʔλͷ෼ੳʹར༻ ςΩετʢࣗಈ຋༁ʣ ΦʔσΟΦʢԻ੠ೝࣝʣ ϏσΦʢΠϝʔδΩϟϓγϣϯʣ

/FVSBM/FUXPSLT <>ΑΓҾ༻

3//T ྫɿөըͷը૾ʹϥϕϧΛ͚͍ͭͨ աڈϑϨʔϜͷ৘ใΛར༻͢Δ ֤ը૾Λݟͨͱ͖ɺϥϕϧΛग़ྗͭͭ͠ɺ஌ࣝΛߋ৽͢ΔϞσϧΛߟ͑Δ <>ΑΓҾ༻

-45.T 3//ͷ໰୊఺ ஌ࣝͷߋ৽͕ແடংʹߦΘΕΔʢ৘ใ͕͙͢ʹมΘΔࣦΘΕΔʣ ௕ظهԱΛ࣋ͭͷ͕ࠔ೉ -45. ௕ظϝϞϦ ๨٫ػߏʢ௕ظهԱ͔ΒԿΛอ࣋͢ΔࣺͯΔ͔൑அʣ อଘػߏʢ৽͍͠σʔλ͔Β͑ΒΕͨ৘ใΛ௕ظهԱʹ௥Ճʣ ϫʔΩϯάϝϞϦ ϑΥʔΧεʢ͍·ඞཁͳ৘ใͷΈΛར༻ʣ

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦͷͲͷ෦෼Λอ࣋͢Δ͔
อ࣋ɺഁغ

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦʹ଍͢৘ใ

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦʹ଍͢৘ใͷ͏ͪɺ
Ͳͷ෦෼͕ॏཁ͔ʁ

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦΛߋ৽

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦͷ͏ͪɺ͍·ॏཁͳ΋ͷ͸ʁ

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ϫʔΩϯάϝϞϦΛநग़

-45.T <>ΑΓҾ༻

આ໌Ͱ༻͍ͨݺশ Ұൠతͳݺশ ௕ظϝϞϦMUNU ηϧঢ়ଶʢDFMMTUBUFʣDU ϫʔΩϯάϝϞϦXNU ӅΕʢIJEEFOTUBUFʣIU ๨٫ػߏSFNFNCFSU ๨٫ήʔτʢGPSHFUHBUFʣGU อଘϕΫτϧTBWFU ೖྗήʔτʢJOQVUHBUFʣJU
ϑΥʔΧεϕΫτϧGPDVTU ग़ྗήʔτʢPVUQVUHBUFʣPU ෼͔Γ΍໊͍͢લͰઆ໌͖͕ͯͨ͠ɺ֤ཁૉ͸ผ໊Ͱݺ͹ΕΔ͜ͱ͕ଟ͍ -45.EJBHSBNT

-45.EJBHSBNT <>ΑΓҾ༻

GU๨٫ήʔτ ϝϞϦʔηϧ$UͷͲͷ෦෼Λอ࣋͢Δ͔ อ࣋ɺഁغ -45.EJBHSBNT <>ΑΓҾ༻

JUɹೖྗήʔτ ৽͍͠ೖྗ͔Β͑ΒΕͨ৘ใͷ͏ͪɺͲ͕͜ॏཁ͔ -45.EJBHSBNT <>ΑΓҾ༻

ηϧঢ়ଶΛߋ৽ -45.EJBHSBNT <>ΑΓҾ༻

ϝϞϦηϧ͔Βग़ྗήʔτΛ௨ͯ͡ɺ ӅΕঢ়ଶΛऔಘ -45.EJBHSBNT <>ΑΓҾ༻

-45.WBSJBOUT ͷ͖ͧ݀ʢQFFQIPMFʣ ֤ήʔτ͔Βηϧͷঢ়ଶΛࢀরͰ͖Δ -45.ʹ͸͞·͟·ͳѥछ͕ଘࡏ͢Δ <>ΑΓҾ༻

-45.WBSJBOUT ๨٫ήʔτͱೖྗήʔτͷಉҰԽ -45.ʹ͸͞·͟·ͳѥछ͕ଘࡏ͢Δ <>ΑΓҾ༻

-45.WBSJBOUT (BUFE3FDVSSFOU6OJU (36 ɹηϧͱӅΕঢ়ଶΛ༥߹ ɹ๨٫ήʔτͱೖྗήʔτΛ༥߹ -45.ʹ͸͞·͟·ͳѥछ͕ଘࡏ͢Δ <>ΑΓҾ༻

-45.ͷ࿦จ঺հ ͔͜͜ΒҎԼͷ࿦จͷ࿩ʹͳΓ·͢ l4FRVFODFUP4FRVFODF7JEFPUP5FYUz 4VCIBTIJOJ7FOVHPQBMBO .BSDVT 3PISCBDI +F⒎%POBIVF 3BZNPOE.PPOFZ 5SFWPS%BSSFMM ,BUF4BFOLP

*OUSPEVDUJPO ϏσΦΩϟϓγϣϯͷ೉͍͠ͱ͜Ζ Մม௕ͷೖྗͱՄม௕ͷग़ྗ ΦϒδΣΫτɺγʔϯɺΞΫγϣϯɺ ଐੑͷଟ༷ੑ ݦஶͳίϯςΩετͷద੾ͳநग़ ϏσΦΫϦοϓͱઆ໌จ͔ΒͳΔσʔληοτΛ༻͍ͯɺϏσΦͷΩϟϓγϣ ϯΛੜ੒͢ΔϞσϧʢ475ʣΛֶश FOEUPFOEͰֶशՄೳͳϞσϧ 4FRVFODFUP4FRVFODFͷֶशͰ࣮੷ͷ͋Δ-45.Λ࢖͏
<>ΑΓҾ༻

3FMBUFE8PSL ॳظͷϏσΦΩϟϓγϣϯͷख๏ εςʔδͷύΠϓϥΠϯ ҙຯͷ͋ΔίϯςΩετͷநग़ʢTVCKFDU WFSC PCKFDUʣ ςϯϓϨʔτΛ࢖ͬͯจষΛੜ੒ ީิͱͳΔPCKFDU BDUJPO TDFOFΛࣝผ͢ΔͨΊʹݸผͷࣝผث͕ඞཁ
ݴޠදݱೳྗ͕े෼Ͱ͸ͳ͍ʢୀ۶ͳจষʹͳΓ͕ͪʣ ࠷৽ͷϏσΦΩϟϓγϣϯͷख๏ $//Λ࢖ͬͯಛ௃நग़ͯ͠ɺͦͷಛ௃ϕΫτϧΛ୯ޠͷγʔέϯεʹσ ίʔυ͢Δ ௕ظґଘؔ܎Λֶश͢Δͷʹదͨ͠-45.͕࢖ΘΕΔ͜ͱ͕ଟ͍

3FMBUFE8PSL ࠷৽ͷϏσΦΩϟϓγϣϯͷख๏4VCIBTIJOJΒʢʣ NFBOQPPMJOHΛ࢖ͬͯɺϏσΦͷ৘ใΛҰͭͷಛ௃ϕΫτϧͰදݱ ϑϨʔϜͷॱংؔ܎Λແࢹ͍ͯ͠ΔͨΊɺ࣌ؒ৘ใΛ׆༻Ͱ͖ͳ͍ <>ΑΓҾ༻

3FMBUFE8PSL ࠷৽ͷϏσΦΩϟϓγϣϯͷख๏-J:BPΒʢʣ ϏσΦΛ 8 ) 5 ͷ̏࣍ݩͰදݱ͠ɺ͔ͦ͜Βಛ௃ྔΛநग़ நग़ͨ͠ಛ௃ྔΛ%$//ʹೖྗ͠ɺBDUJPOΛࣝผ BUUFOUJPOΛ࢖ͬͯϑϨʔϜͷಛ௃ྔͷॏΈΛௐ੔ ࣌ؒ࣠ͷಛ௃நग़
BUUFOUJPOʹΑͬͯύϑΥʔϚϯε޲্ʹ੒ޭ <>ΑΓҾ༻

"QQSPBDI ೖྗϑϨʔϜγʔέϯε ग़ྗ୯ޠγʔέϯε ҎԼͷ৚͖݅ͭ֬཰Λਪఆ͢ΔϞσϧΛߟ͑Δ ೖग़ྗͷγʔέϯε͕Մม௕͚ͩͲͲ͏΍ͬͯରॲ͢Δʁ ϑϨʔϜγʔέϯεΛͭͣͭΤϯίʔυ͠ɺϏσΦΛҰͭͷજࡏม਺දݱͰද͢ જࡏม਺ϕΫτϧΛ୯ޠͷγʔέϯεʹσίʔυ͢Δ ͭલͷ୯ޠ ϏσΦͷજࡏม਺දݱͷ࠷৽ঢ়ଶ

"QQSPBDI ݸͷӅΕϢχοτΛ࣋ͭͭͷ-45.ΛελοΫ͢Δ ੺৭ͷ-45.ϨΠϠʔ͸ը૾ͷΤϯίʔυΛ୲౰ ྘৭ͷ-45.ϨΠϠʔ͸୯ޠ΁ͷσίʔυΛ୲౰ QBE͸θϩϕΫτϧɺ#04͸CFHJOJOHPGTFOUFODFλάɺ&04͸FOEPGTFOUFODF λά ҎԼͷର਺໬౓Λ࠷େԽ͢ΔΑ͏ʹύϥϝʔλΛֶश ͨͩ͠ σίʔμ-45.ͷग़ྗ <>ΑΓҾ༻

7JEFPBOE5FYU3FQSFTFOUBUJPO 3(#GSBNFT $//Ͱಛ௃நग़ *NBHF/FUEBUBTFUͷը૾ͰֶशࡁΈͷ$// 0QUJDBM'MPX BDUJWJUZͷࣝผʹ༗ޮͰ͋ΔͱݴΘΕ͍ͯΔ qPXJNBHFTΛ$//ʹೖྗ͠ಛ௃ྔΛੜ੒ 0QUJDBM'MPXΛBDUJWJUZࣝผ͢ΔͨΊͷσʔληοτʢ6$'WJEFPEBUBTFUʣ ͰֶशࡁΈͷ$//Λ࢖ͬͨ ্هͭͷಛ௃Λ࢖ͬͯ͑ΒΕͨ݁ՌΛ݁߹ͨ͠΋ͷΛ࠷ऴతͳείΞͱ͢Δ
ϏσΦͷಛ௃ྔ ςΩετͷಛ௃ྔ POFIPUFODPEJOHʢPG/DPEJOHʣΛઢܗม׵ͯ͠ɺ࣍ݩʹͨ͠΋ͷ

&YQFSJNFOU <>ΑΓҾ༻

"QQFOEJY0QUJDBM'MPX ࿈ଓ͢ΔϑϨʔϜ͔Β෺ମͷҠಈ৘ใΛநग़͢Δख๏ <>ΑΓҾ༻ <>ΑΓҾ༻

3FGFSFODFT <>&YQMPSJOH-45.T <>6OEFSTUBOEJOH-45./FUXPSLT <>4FRVFODFUP4FRVFODFr7JEFPUP5FYU <>5SBOTMBUJOH7JEFPTUP/BUVSBM-BOHVBHF6TJOH%FFQ3FDVSSFOU/FVSBM/FUXPSLT <>%FTDSJCJOH7JEFPTCZ&YQMPJUJOH5FNQPSBM4USVDUVSF <>0QUJDBM'MPX

Introduction to LSTM

Introduction to LSTM

KenjiH

More Decks by KenjiH

Other Decks in Technology

Featured

Transcript

-45.ೖ໳ Kenji Hayashida

-45.ͱ͸ MPOHTIPSUUFSNNFNPSZ SFDVSSFOUOFVSBMOFUXPSLNPEFMͷҰछ γʔέϯεσʔλͷ෼ੳʹར༻ ςΩετʢࣗಈ຋༁ʣ ΦʔσΟΦʢԻ੠ೝࣝʣ ϏσΦʢΠϝʔδΩϟϓγϣϯʣ

/FVSBM/FUXPSLT <>ΑΓҾ༻

3//T ྫɿөըͷը૾ʹϥϕϧΛ͚͍ͭͨ աڈϑϨʔϜͷ৘ใΛར༻͢Δ ֤ը૾Λݟͨͱ͖ɺϥϕϧΛग़ྗͭͭ͠ɺ஌ࣝΛߋ৽͢ΔϞσϧΛߟ͑Δ <>ΑΓҾ༻

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦͷͲͷ෦෼Λอ࣋͢Δ͔

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦʹ଍͢৘ใ

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦʹ଍͢৘ใͷ͏ͪɺ

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦΛߋ৽

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ௕ظϝϞϦͷ͏ͪɺ͍·ॏཁͳ΋ͷ͸ʁ

-45.T ࣌ࠁͷͱ͖ೖྗɹ͓Αͼ લ࣌ࠁͷ௕ظϝϞϦɹɹɹɹɺϫʔΩϯάϝϞϦ͕༩͑ΒΕΔ xt ltmt 1 wmt 1 t ϫʔΩϯάϝϞϦΛநग़

-45.T <>ΑΓҾ༻

આ໌Ͱ༻͍ͨݺশ Ұൠతͳݺশ ௕ظϝϞϦMUNU ηϧঢ়ଶʢDFMMTUBUFʣDU ϫʔΩϯάϝϞϦXNU ӅΕʢIJEEFOTUBUFʣIU ๨٫ػߏSFNFNCFSU ๨٫ήʔτʢGPSHFUHBUFʣGU อଘϕΫτϧTBWFU ೖྗήʔτʢJOQVUHBUFʣJU

-45.EJBHSBNT <>ΑΓҾ༻

GU๨٫ήʔτ ϝϞϦʔηϧ$UͷͲͷ෦෼Λอ࣋͢Δ͔ อ࣋ɺഁغ -45.EJBHSBNT <>ΑΓҾ༻

JUɹೖྗήʔτ ৽͍͠ೖྗ͔Β͑ΒΕͨ৘ใͷ͏ͪɺͲ͕͜ॏཁ͔ -45.EJBHSBNT <>ΑΓҾ༻

ηϧঢ়ଶΛߋ৽ -45.EJBHSBNT <>ΑΓҾ༻

ϝϞϦηϧ͔Βग़ྗήʔτΛ௨ͯ͡ɺ ӅΕঢ়ଶΛऔಘ -45.EJBHSBNT <>ΑΓҾ༻

-45.WBSJBOUT ͷ͖ͧ݀ʢQFFQIPMFʣ ֤ήʔτ͔Βηϧͷঢ়ଶΛࢀরͰ͖Δ -45.ʹ͸͞·͟·ͳѥछ͕ଘࡏ͢Δ <>ΑΓҾ༻

-45.WBSJBOUT ๨٫ήʔτͱೖྗήʔτͷಉҰԽ -45.ʹ͸͞·͟·ͳѥछ͕ଘࡏ͢Δ <>ΑΓҾ༻

-45.WBSJBOUT (BUFE3FDVSSFOU6OJU (36 ɹηϧͱӅΕঢ়ଶΛ༥߹ ɹ๨٫ήʔτͱೖྗήʔτΛ༥߹ -45.ʹ͸͞·͟·ͳѥछ͕ଘࡏ͢Δ <>ΑΓҾ༻

-45.ͷ࿦จ঺հ ͔͜͜ΒҎԼͷ࿦จͷ࿩ʹͳΓ·͢ l4FRVFODFUP4FRVFODF7JEFPUP5FYUz 4VCIBTIJOJ7FOVHPQBMBO .BSDVT 3PISCBDI +F⒎%POBIVF 3BZNPOE.PPOFZ 5SFWPS%BSSFMM ,BUF4BFOLP

3FMBUFE8PSL ॳظͷϏσΦΩϟϓγϣϯͷख๏ εςʔδͷύΠϓϥΠϯ ҙຯͷ͋ΔίϯςΩετͷநग़ʢTVCKFDU WFSC PCKFDUʣ ςϯϓϨʔτΛ࢖ͬͯจষΛੜ੒ ީิͱͳΔPCKFDU BDUJPO TDFOFΛࣝผ͢ΔͨΊʹݸผͷࣝผث͕ඞཁ

3FMBUFE8PSL ࠷৽ͷϏσΦΩϟϓγϣϯͷख๏4VCIBTIJOJΒʢʣ NFBOQPPMJOHΛ࢖ͬͯɺϏσΦͷ৘ใΛҰͭͷಛ௃ϕΫτϧͰදݱ ϑϨʔϜͷॱংؔ܎Λແࢹ͍ͯ͠ΔͨΊɺ࣌ؒ৘ใΛ׆༻Ͱ͖ͳ͍ <>ΑΓҾ༻

3FMBUFE8PSL ࠷৽ͷϏσΦΩϟϓγϣϯͷख๏-J:BPΒʢʣ ϏσΦΛ 8 ) 5 ͷ̏࣍ݩͰදݱ͠ɺ͔ͦ͜Βಛ௃ྔΛநग़ நग़ͨ͠ಛ௃ྔΛ%$//ʹೖྗ͠ɺBDUJPOΛࣝผ BUUFOUJPOΛ࢖ͬͯϑϨʔϜͷಛ௃ྔͷॏΈΛௐ੔ ࣌ؒ࣠ͷಛ௃நग़

&YQFSJNFOU <>ΑΓҾ༻

"QQFOEJY0QUJDBM'MPX ࿈ଓ͢ΔϑϨʔϜ͔Β෺ମͷҠಈ৘ใΛநग़͢Δख๏ <>ΑΓҾ༻ <>ΑΓҾ༻

3FGFSFODFT <>&YQMPSJOH-45.T <>6OEFSTUBOEJOH-45./FUXPSLT <>4FRVFODFUP4FRVFODFr7JEFPUP5FYU <>5SBOTMBUJOH7JEFPTUP/BUVSBM-BOHVBHF6TJOH%FFQ3FDVSSFOU/FVSBM/FUXPSLT <>%FTDSJCJOH7JEFPTCZ&YQMPJUJOH5FNQPSBM4USVDUVSF <>0QUJDBM'MPX