Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning Book 10その2 / deep learning book 1...

himkt
January 29, 2018

Deep Learning Book 10その2 / deep learning book 10 vol2

himkt

January 29, 2018
Tweet

More Decks by himkt

Other Decks in Research

Transcript

  1. &DIP4UBUF/FUXPSLT w 3//ʹֶ͓͍ͯश͕େมͳύϥϝʔλ w ӅΕ૚ӅΕ૚ SFDVSSFOUXFJHIUT  w ೖྗӅΕ૚ JOQVUXFJHIUT

     w &DIP4UBUF/FUXPSL w ӅΕ૚ӅΕ૚͸ॏΈΛݻఆ w ֶश͢Δͷ͸ʜ w ೖྗӅΕ૚ JOQVUXFJHIUT  w ӅΕ૚ग़ྗ PVUQVUXFJHIUT 
  2. ,FSOFMNBDIJOFͱͷྨࣅੑ w Χʔωϧ͕΍ͬͯΔ͜ͱͬͯʁ w ೚ҙͷ௕͞ͷܥྻΛݻఆ௕ͷϕΫτϧ΁ࣸ͢ w ݻఆ௕ͷϕΫτϧΛ༻͍ͯ෼ྨث͕໰୊Λղ͘ w ͜ͷܗͷ৔߹ɼֶशͷج४ͷઃܭ͕༰қͰ͋Δ w

    ग़ྗઢܗճؼͷ৔߹͸.4&ͰֶशͰ͖Δ w &4/T΋ೖྗΛԿΒ͔ͷϕΫτϧʹࣸ͢ૢ࡞Λ͍ͯ͠Δ w தͷॏΈ͸ݻఆ͍ͯ͠Δ  ͍͔ʹաڈͷ৘ใΛ๛෋ʹؚΉදݱ͕ಘΒΕΔ
 ॏΈΛઃఆ͢Ε͹Α͍͔ʁ શવҙຯ͕Θ͔Βͣʜ  3//ΛಈతγεςϜͱΈͳ͢ γεςϜ͕҆ఆ͢ΔΑ͏ͳॏΈΛઃఆ͢Δ
  3. -FBLZ6OJUTBOE0UIFS4USBUFHJFTGPS.VMUJQMF5JNF4DBMF w աڈͷ৘ใΛ఻͑ΔͨΊͷ޻෉  "EEJOH4LJQ$POOFDUJPOTUISPVHI5JNF w ޯ഑ͷফࣦͷ଎౓͕஗͘ͳΔ w രൃ͸ݩͷ3//ͱಉ͡଎౓Ͱൃੜ͢Δ 

    -FBLZ6OJUTBOEB4QFDUSVNPG%J⒎FSFOU5JNF4DBMFT w աڈͷ৘ใΛͲͷఔ౓࢒͔͢Λ੍ޚ͢Δ  3FNPWJOH$POOFDUJPOT w ୹͍࣌ࠁͰͷґଘΛ௕͍࣌ࠁͰͷґଘʹஔ͖׵͑Δ 
  4. -FBLZ6OJUT w աڈͷ৘ใΛͲͷ͘Β͍࢒͔͢Λௐ੔͢Δ w ҠಈฏۉͷΑ͏ͳ;Δ·͍Λ͢Δ w Ћ͕େ͖͍ ʹ͍ۙ աڈͷ৘ใΛΑΓอଘ͢Δ w

    Ћ͕খ͍͞ ʹ͍ۙ աڈͷ৘ใΛ͙͢ʹࣺͯΔ w Ћ͸ద౰ʹܾఆ͢ΔϋΠύʔύϥϝʔλ  µ(t) ↵µ(t 1) + (1 ↵)v(t)
  5. ਺ࣜతʹൺֱ͢ΔʢόΠΞεΛແࢹʣ  -45. (36 zt = (xtUz + ht 1Wz)

    rt = (xtUr + ht 1Wr) ˜ ht = tanh ⇣ xt + Uh + (rt ht 1)Wh ⌘ ht = (1 zt) ht 1 + zt ˜ ht it = (xtUi + ht 1Wi) ft = (xtUf + ht 1Wf ) ot = (xtUo + ht 1Wg) ˜ Ct = tanh (xtUg + ht 1Wg) Ct = (ft Ct 1 + it ˜ Ct) ht = tanh (Ct) ot (36Ͱ͸ೖྗήʔτͱ๨٫ήʔτ͕౷߹͞Ε͍ͯΔ
  6. $MJQQJOH(SBEJFOU w ޯ഑๏ϕʔεͷख๏ʹΑΔͱʜ w ֑ͷपΓͰ఺͕ਧ͖ඈΜͰ͠·͏ ޯ഑രൃ  w ޯ഑͕େ͖͘ͳΓ͗ͨ͢Βޯ഑ͷϊϧϜͰׂΔ w

    ޯ഑ΛHͱͯ͠ʜ w W͸ϋΠύʔύϥϝʔλ ࣗવݴޠॲཧͩͱ͕ଟ͍  g ( gv ||g|| (||g|| > v) g (otherwise)
  7. &YQMJDJU.FNPSZ w χϡʔϥϧωοτϫʔΫ͸ʜ w ҉໧తͳ৘ใͷอ࣋͸ಘҙ w ໌ࣔతͳ৘ใ ࣄ࣮ ͷอ࣋͸ۤख w

    ໌ࣔతͳ৘ใΛอ࣋͠ɼਪ࿦ʹ׆༻͢Δߏ଄
 ʢϫʔΩϯάϝϞϦͷಋೖʣ w .FNPSZ/FUXPSLT w /FVSBM5VSJOH.BDIJOF 
  8. "TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ w ਖ਼֬ͳϝϞϦͷΞυϨεΛग़ྗ͢Δͷ͸೉͍͠ w ଟ͘ͷϝϞϦηϧͷॏΈ෇͖ฏۉΛͱΔ w ॏΈ͸ιϑτϚοΫεͳͲͰ࡞Δ
 ʢͰ͖Δ͚ͩҰՕॴͷϝϞϦΛࢀর͢ΔΑ͏ʹʣ w ϝϞϦηϧ͸εΧϥΑΓ΋ϕΫτϧͷํ͕ྑ͍

    w ίϯςϯπϕʔεΞυϨογϯά͕ՄೳʹͳΔ w ʮl8FBMMMJWFJOBZFMMPXTVCNBSJOFzΛؚΉՎࢺΛݟ͚ͭΔʯ w ʢϩέʔγϣϯϕʔεΞυϨογϯάͱ͸ʁʣ w ʮεϩοτ347ʹ֨ೲ͞Ε͍ͯΔՎࢺΛऔಘ͢Δʯ w ʢΞυϨογϯά͸Ξςϯγϣϯͱಉ͡ܗࣜʣ