Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

A Natural Policy Gradient and TRPO by Andrew Ga...

Avatar for Tokyo Machine Learning Society Tokyo Machine Learning Society
October 28, 2016
890
1

A Natural Policy Gradient and TRPO by Andrew Gambardella - RRS #1

Avatar for Tokyo Machine Learning Society

Tokyo Machine Learning Society

October 28, 2016

More Decks by Tokyo Machine Learning Society

See All by Tokyo Machine Learning Society
Natural Gradient Learning by Shun'ichi Amari, RIKEN - RRS #1
Avatar for Tokyo Machine Learning Society tmls
1
460
Generative Adversarial Networks by Hasanov Vagif - TMLS #4
Avatar for Tokyo Machine Learning Society tmls
3
550
Character-Level Machine Translation with Bi-scale RNNs by Joe Bullard - TMLS #4
Avatar for Tokyo Machine Learning Society tmls
2
440
NVIDIA Deep Learning Platform and latest information by Takeshi Izaki, Nvidia - TMLS #4　
Avatar for Tokyo Machine Learning Society tmls
1
380
Neural Translation by David and Stefano - TMLS#3
Avatar for Tokyo Machine Learning Society tmls
3
360
Extending Cloud Vision API by Ray Sakai -TMLS#3
Avatar for Tokyo Machine Learning Society tmls
2
220
Bayesian Optimization with TensorFlow/Keras by Keisuke Kamataki - TMLS #2
Avatar for Tokyo Machine Learning Society tmls
1
9.3k

Featured

See All Featured
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
Avatar for Andy Polaine apolaine
2
500
Making the Leap to Tech Lead
Avatar for Ryan Cromwell cromwellryan
135
10k
Digital Ethics as a Driver of Design Innovation
Avatar for Per Axbom axbom
PRO
1
480
Gemini Prompt Engineering: Practical Techniques for Tangible AI Outcomes
Avatar for Mfonobong Umondia mfonobong
2
610
Faster Mobile Websites
Avatar for Dean Hume deanohume
310
32k
Principles of Awesome APIs and How to Build Them.
Avatar for Keavy McMinn keavy
128
18k
Heart Work Chapter 1 - Part 1
Avatar for Lake Fama lfama
PRO
10
37k
XXLCSS - How to scale CSS and keep your sanity
Avatar for Zaharenia Atzitzikaki sugarenia
250
1.3M
Taking LLMs out of the black box: A practical guide to human-in-the-loop distillation
Avatar for Ines Montani inesmontani
PRO
3
2.5k
Visual Storytelling: How to be a Superhuman Communicator
Avatar for David Neal reverentgeek
2
740
Bridging the Design Gap: How Collaborative Modelling removes blockers to flow between stakeholders and teams @FastFlow conf
Avatar for Kenny Baas-Schwegler baasie
0
750
Easily Structure & Communicate Ideas using Wireframe
Avatar for Afnizar Nur Ghifari afnizarnur
194
17k

Transcript

  1. "/BUVSBM1PMJDZ(SBEJFOU
 
 5SVTU3FHJPO1PMJDZ0QUJNJ[BUJPO "OESFX(BNCBSEFMMB
 3FBDUJWF3FTFBSDI4FNJOBS

  2. 3FJOGPSDFNFOU-FBSOJOH

  3. "QQSPBDIFT ↟ 7BMVFCBTFE ↟ 1PMJDZCBTFE ↟ -FBSOBQPMJDZ BDUPS BOETUBUFWBMVFGVODUJPO DSJUJD

     KPJOUMZ ↟ 0UIFST

  4. %FFQ3FJOGPSDFNFOU-FBSOJOH

  5. #VUXIZMFBSOB2GVODUJPO ↟ *ONBDIJOFMFBSOJOH XFPQUJNJ[FGPSUIFUIJOHXFDBSF BCPVU ↟ *OUIJTDBTF JUsTUIFFYQFDUFEUPUBMSFXBSEPGUIFQPMJDZ ↟ $BOFNQJSJDBMMZTIPXUIBU%2/JTOPUWFSZHPPE

    DPNQBSFEUPQPMJDZHSBEJFOUBOEBDUPSDSJUJDNFUIPET TFF"TZODISPOPVT.FUIPETGPS%FFQ3FJOGPSDFNFOU -FBSOJOH

  6. 1PMJDZ(SBEJFOU

  7. 3&*/'03$&FTUJNBUPS
 BMTPLOPXOBTMJLFMJIPPESBUJPFTUJNBUPSTDPSF GVODUJPOFTUJNBUPS -FUYCFBSBOEPNWBSJBCMFXJUIY_Q ↟Ƕ 5IFO

  8. .BOZEJGGFSFOUDIPJDFTGPSG

  9. 1SPTPGQPMJDZHSBEJFOUNFUIPET DPNQBSFEUPEFFQ2MFBSOJOH ↟ 0QUJNJ[JOHGPSXIBUXFBDUVBMMZXBOUnNBYJNVN FYQFDUFEUPUBMSFXBSEPGUIFQPMJDZ ↟ #FUUFSGPSIJHIEJNFOTJPOBMUBTLT ↟ #FUUFSGPSDPOUJOVPVTUBTLT NBQZPVSTUBUFUPUIFNFBO

    BOEWBSJBODFPGB(BVTTJBOEJTUSJCVUJPO  ↟ $BOCPPUTUSBQPGGPGQPMJDJFTDSFBUFECZJNJUBUJPOMFBSOJOH ↟ $BOMFBSOBTUPDIBTUJDQPMJDZ HPPEJGTPNFTUBUFTBSF BMJBTFE

  10. $POTPGQPMJDZHSBEJFOUNFUIPET DPNQBSFEUPEFFQ2MFBSOJOH ↟ %2/NPSFTBNQMFFGGJDJFOUPODFSUBJOUBTLT

  11. $POTPGQPMJDZHSBEJFOU NFUIPETJOHFOFSBM ↟ 'BJSMZEJGGJDVMUUPEFCVH ↟ &TUJNBUPSPGUIFHSBEJFOUIBTIJHIWBSJBODF ↟ $POWFSHFUPBMPDBMNJOJNVN DBOsUUSBJOCJHOFVSBMOFUT CFDBVTFPGQSFWJPVTQPJOUBOZHVBSBOUFFTBCPVUMPDBM

    NJOJNBPGFYUSFNFMZIJHIEJNFOTJPOBMGVODUJPOTEPOPU IPME

  12. )PXUPEFBMXJUIFTUJNBUPS WBSJBODF ↟ $FSUBJOUSJDLTDBOIFMQVT 4FF3FJOGPSDFNFOU-FBSOJOH /FVSBM5VSJOH.BDIJOFT PSTFF-FBSOJOH0OMJOF "MJHONFOUTXJUI$POUJOVPVT3FXBSET1PMJDZ(SBEJFOU  ↟

    8FDBOBMTPEPHSBEJFOUEFTDFOUVTJOHBEJSFDUJPOBOE PSTUFQTJ[FUIBUNBLFTNPSFTFOTFGPS3-UIBOWBOJMMB 4(%

  13. /BUVSBM1PMJDZ(SBEJFOU ↟ 6TF"NBSJsTUIFPSZUPEFGJOFBNFUSJDCBTFEOPUPOUIFDIPJDF PGDPPSEJOBUFT CVUPOUIFNBOJGPMEUIBUUIFTFDPPSEJOBUFT QBSBNFUFSJ[F ↟ "TTIPXOCZ"NBSJ UIF'JTIFSJOGPSNBUJPONBUSJYJTBO JOWBSJBOUNFUSJDPOUIFTQBDFPGQBSBNFUFSTPGQSPCBCJMJUZ

    EJTUSJCVUJPOT*UEFGJOFTUIFTBNFEJTUBODFCFUXFFOUXPQPJOUT SFHBSEMFTTPGUIFDIPJDFPGDPPSEJOBUFT JFUIF QBSBNFUFSJ[BUJPO  ↟ 5IJTHJWFTTUFFQFTUEFTDFOUEJSFDUJPO(− Ƕ ˜ǵ Ƕ XIFSF˜ǵ Ƕ  JTUIFEFTDFOUEJSFDUJPOHJWFOCZUIFtWBOJMMBuQPMJDZHSBEJFOU

  14. 8IZVTFUIFOBUVSBMHSBEJFOU GPS3- ↟ 8FXBOUUIFTUBUFTXFTFFXJUIPVSOFXQPMJDZUPCF TJNJMBSUPUIPTFXFTBXXJUIPVSPMEQPMJDZ TPXFDBO SFVTFLOPXMFEHFGSPNQBTUSPMMPVUT ↟ #ZDPOTUSBJOJOHPVSOFXQPMJDZUPCFtDMPTFuUPPVSPME QPMJDZJOUIJTXBZ

    XFDBODPOUSPMUIFBDUJPOEJTUSJCVUJPO DIBOHFBTBQSPYZ

  15. 8IZVTFUIFOBUVSBMHSBEJFOU GPS3- ↟ *GXFUBLFBTNBMMTUFQJO&VDMJEFBOQBSBNFUFSTQBDF  FWFOBTNBMMTUFQDPVMEDBVTFXJMEDIBOHFTJOUIF BDUJPOEJTUSJCVUJPO*UJTUIFSFGPSFFTTFOUJBMUIBUXFUBLF TUFQTJOUIFOBUVSBMHSBEJFOUEJSFDUJPO ↟ "MTP

    IPXDBOXFDIPPTFBOBQQSPQSJBUFTUFQTJ[F  XIFOUIFTUBUJTUJDTPGPVSEBUBBSFDPOTUBOUMZDIBOHJOH  BTPVSQPMJDZDIBOHFT TPEPFTPVSEBUB
  16. None

  17. "TMJHIUJTTVF ↟ 5IFOBUVSBMQPMJDZHSBEJFOUNVTUGPSNBOEJOWFSUUIF FNQJSJDBM'*. ↟ 5IJTJTUPPFYQFOTJWF

  18. 5SVODBUFE/BUVSBM1PMJDZ (SBEJFOU 5/1( ↟ *OTUFBEPGDPNQVUJOHUIF'*. DPNQVUFUIFTBNF EJSFDUJPOPGOBUVSBMQPMJDZHSBEJFOUVTJOHBDPOKVHBUF HSBEJFOUBMHPSJUINUIBUPOMZSFRVJSFTDPNQVUJOHBO BQQSPYJNBUJPOPG( Ƕ

    WGPSBSCJUSBSZWFDUPSW

  19. 5SVTU3FHJPO1PMJDZ 0QUJNJ[BUJPO 5310 ↟ $POTUSBJOUIF,-EJWFSHFODFCFUXFFOPVSPMEQPMJDZ BOEPVSOFXQPMJDZUPCFXJUIJOTPNFUSVTUSFHJPO SBEJVTDz ↟ #PVOEPO,-EJWFSHFODFJTBMTPBCPVOEPOIPXNVDI PVSQFSGPSNBODFDBOEFHSBEFXJUIJOUIJTPOFTUFQ

  20. 5/1(BOE5310 ↟ 5/1(DBOCFPCUBJOFEBTBTQFDJBMDBTFPG5310XJUIB MJOFBSBQQSPYJNBUJPOUP-BOEBRVBESBUJDBQQSPYJNBUJPO UPUIF%,- DPOTUSBJOU/PUJDFTUFQTJ[FǹJO5/1(
 
 
 
 


    
 
 
 5310 5/1(XBQQSPYJNBUJPO
  21. None

  22. "OEOPXGPSTPNFUIJOH DPNQMFUFMZEJGGFSFOU

  23. (FOFSBUJWF"EWFSTBSJBM /FUXPSLT ↟ "SFDFOUNFUIPEGPSVOTVQFSWJTFEMFBSOJOH ↟ (FOFSBUPS(TBNQMFTGSPNBEJTUSJCVUJPOBOEEJTDSJNJOBUPS% DMBTTJGJFTTBNQMFTBTSFBMPSGBMTF ↟ (JTUSZJOHUPtUSJDLu%CZJODSFBTJOHUIFQSPCBCJMJUZUIBU%XJMM DMBTTJGZ(sTTBNQMFTBTSFBM%JTUSZJOHUPNJOJNJ[FUIF

    QSPCBCJMJUZPGCFJOHUSJDLFE ↟ 5IJTHJWFTVTUIFBEWFSTBSJBMMPTTGVODUJPO
 
 


  24. 8IBUEPFTUIJTIBWFUP EPXJUI3-

  25. *NJUBUJPO-FBSOJOH ↟ *NJUBUJPOMFBSOJOHTFUUJOH(JWFOUSBKFDUPSJFTGSPNBO FYQFSU MFBSOBQPMJDZ ↟ 3FHVMBSTVQFSWJTFEMFBSOJOH CFIBWJPSBMDMPOJOH XJMM BMXBZTXPSLJOUIFJOGJOJUFEBUBSFHJNF

    ↟ $BOXFNBLFTPNFUIJOHNPSFTBNQMFFGGJDJFOU  LOPXJOHUIBUXFBSFMPPLJOHGPSBO.%1QPMJDZ

  26. (FOFSBUJWF"EWFSTBSJBM *NJUBUJPO-FBSOJOH ↟ *NJUBUJPOMFBSOJOHDBOCFGPSNVMBUFEBTBNJOJNBYQSPCMFN XIFSFUIFBJNJTUPMFBSOUIFDPTUGVODUJPOUIBUNJOJNJ[FT UIFBNPVOUCZXIJDIUIFPQUJNBMQPMJDZPVUQFSGPSNTUIF FYQFSUQPMJDZ ↟ #VUMFBSOJOHBDPTUGVODUJPOJTFRVJWBMFOUUPNJOJNJ[JOHUIF SFHVMBSJ[FEEJTUBODFCFUXFFOTUBUFBDUJPOPDDVQBODZ

    EJTUSJCVUJPOTGPSUIFFYQFSUQPMJDZBOEMFBSOFEQPMJDZ ↟ #ZVTJOHBEFFQOFVSBMOFUXPSLXJUIUIFDSPTTFOUSPQZMPTT BTUIFEJTUBODFCFUXFFOPDDVQBODZEJTUSJCVUJPOT XFDBO GPSHPMFBSOJOHBDPTUGVODUJPODPNQMFUFMZ

  27. (FOFSBUJWF"EWFSTBSJBM *NJUBUJPO-FBSOJOH

  28. None

  29. 3FTPVSDFT ↟ %FFQ3FJOGPSDFNFOU-FBSOJOH1POHGSPN1JYFMT "OESFK ,BSQBUIZsTCMPH ↟ -FDUVSF1PMJDZ(SBEJFOU %BWJE4JMWFST3-$PVSTF ↟ %FFQ3FJOGPSDFNFOU-FBSOJOH"5VUPSJBM

    0QFO"* ↟ 1PMJDZ(SBEJFOU.FUIPET 4DIPMBSQFEJB

  30. 3FGFSFODFT ↟ "NBSJ 4/BUVSBMHSBEJFOUXPSLTFGGJDJFOUMZJOMFBSOJOH/FVSBM$PNQVUBUJPO      ↟

    %VBO: $IFO9 )PVUIPPGU3 4DIVMNBO+ BOE"CCFFM1#FODINBSLJOHEFFQ SFJOGPSDFNFOUMFBSOJOHGPSDPOUJOVPVTDPOUSPMBS9JWQSFQSJOUBS9JW  ↟ )P + BOE&SNPO 4(FOFSBUJWFBEWFSTBSJBMJNJUBUJPOMFBSOJOHBS9JWQSFQSJOUBS9JW   ↟ ,BLBEF 4"OBUVSBMQPMJDZHSBEJFOU*O"EWBODFTJO/FVSBM*OGPSNBUJPO1SPDFTTJOH 4ZTUFNT QQm.*51SFTT  ↟ -VP: $IJV$ +BJUMZ. BOE4VUTLFWFS*-FBSOJOHPOMJOFBMJHONFOUTXJUIDPOUJOVPVT SFXBSETQPMJDZHSBEJFOUBS9JWQSFQSJOUBS9JW  ↟ .OJI7 #BEJB" .JS[B. (SBWFT" -JMMJDSBQ5 )BSMFZ5 4JMWFS% BOE,BWVLDVPHMV, "TZODISPOPVTNFUIPETGPSEFFQSFJOGPSDFNFOUMFBSOJOHBS9JWQSFQSJOUBS9JW  

  31. 3FGFSFODFT ↟ .OJI 7 ,BWVLDVPHMV , 4JMWFS % (SBWFT "

    "OUPOPHMPV * 8JFSTUSB % BOE 3JFENJMMFS .1MBZJOH"UBSJXJUIEFFQSFJOGPSDFNFOUMFBSOJOHBS9JWQSFQSJOUBS9JW   ↟ 1GBV % BOE7JOZBMT 0$POOFDUJOHHFOFSBUJWFBEWFSTBSJBMOFUXPSLTBOEBDUPS DSJUJDNFUIPETBS9JWQSFQSJOUBS9JW  ↟ 4DIVMNBO + -FWJOF 4 .PSJU[ 1 +PSEBO . BOE"CCFFM 15SVTUSFHJPOQPMJDZ PQUJNJ[BUJPOBS9JWQSFQSJOUBS9JW  ↟ 4DIVMNBO + .PSJU[ 1 -FWJOF 4 +PSEBO . BOE"CCFFM 1)JHIEJNFOTJPOBM DPOUJOVPVTDPOUSPMVTJOHHFOFSBMJ[FEBEWBOUBHFFTUJNBUJPOBS9JWQSFQSJOUBS9JW   ↟ ;BSFNCB 8 BOE4VUTLFWFS *3FJOGPSDFNFOUMFBSOJOHOFVSBMUVSJOHNBDIJOFT BS9JWQSFQSJOUBS9JW 

  32. 2VFTUJPOT