Upgrade to Pro — share decks privately, control downloads, hide ads and more …

20170419-TFUG

 20170419-TFUG

4月19日のTensorFlow User Groupの発表資料です。

ARIYAMA Keiji

April 19, 2017
Tweet

More Decks by ARIYAMA Keiji

Other Decks in Technology

Transcript

  1. ৞ΈࠐΈΦʔτΤϯίʔμʔ  conv1 9x9x3 64 padding: SAME conv2 9x9x64 128

    conv4 5x5x128 64 conv3-0 7x7x128 128 conv3-1 7x7x128 128 dconv1 9x9x64 3 padding: SAME dconv2 9x9x128 64 dconv4 5x5x64 128 dconv3-1 7x7x128 128 dconv3-2 7x7x128 128 conv3-2 7x7x128 128 dconv3-2 7x7x128 128 256x256 256x256
  2. Ϛϧν(16 UPXFS@HSBET<> GPSJJOSBOHF '-"(4OVN@HQVT 
 XJUIUGEFWJDF HQVEJ 
 XJUIUGOBNF@TDPQF T@E

    508&3@/".& J BTTDPQF
 MPTT@@UPXFS@MPTT TDPQF GJMF@MJTU 
 .0%&-*/165@4*;& .0%&-065165@4*;& 
 '-"(4CBUDI@TJ[F 
 
 UGTVNNBSZTDBMBS UPUBM@MPTT MPTT 
 
 UGHFU@WBSJBCMF@TDPQF SFVTF@WBSJBCMFT 
 
 HSBETPQUDPNQVUF@HSBEJFOUT MPTT 
 UPXFS@HSBETBQQFOE HSBET 
  3. Ϛϧν(16 BWH@HSBET@@BWFSBHF@HSBEJFOUT UPXFS@HSBET 
 
 BQQMZ@HSBEJFOU@PQ@@BQQMZ@HSBEJFOUT BWH@HSBET PQU HMPCBM@TUFQ 


    
 WBSJBCMF@BWFSBHFTUGUSBJO&YQPOFOUJBM.PWJOH"WFSBHF  
 HMPCBM@TUFQ 
 WBSJBCMFT@BWFSBHFT@PQ WBSJBCMF@BWFSBHFTBQQMZ UGUSBJOBCMF@WBSJBCMFT 
 
 USBJO@PQUGHSPVQ BQQMZ@HSBEJFOU@PQ WBSJBCMFT@BWFSBHFT@PQ 
  4. શମͷॲཧ TOWER ޯ഑ ʢgradsʣ TOWER ޯ഑ ʢgradsʣ TOWER ޯ഑ ʢgradsʣ

    TOWER ޯ഑ ʢgradsʣ ޯ഑ͷฏۉΛܭࢉ
 __average_gradients શମͷޯ഑ ʢavg_gradsʣ ࠷దԽ
 __apply_gradients params params params params
  5. ৞ΈࠐΈΦʔτΤϯίʔμʔ  conv2 9x9x64 128 conv4 5x5x128 64 conv3-0 7x7x128

    128 conv3-1 7x7x128 128 dconv2 9x9x128 64 dconv4 5x5x64 128 dconv3-1 7x7x128 128 dconv3-2 7x7x128 128 conv3-2 7x7x128 128 dconv3-2 7x7x128 128 256x256 256x256 conv1 9x9x3 64 padding: SAME dconv1 9x9x64 3 padding: SAME
  6. E tensorflow/stream_executor/cuda/cuda_dnn.cc:347] Loaded runtime CuDNN library: 4007 (compatibility version 4000)

    but source was compiled with 5103 (compatibility version 5100).  ύλʔϯ#ɿ5FOTPS'MPXͷΞοϓσʔτ https://github.com/tensorflow/tensorflow/issues/4251
  7. ஗͍ʜʜ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ

    4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ  128 sample / batch learning rate 0.001
  8. 

  9. 

  10.  Hello, Thank you for requesting additional quota. To help

    us ensure that this is a legitimate request and that the resources you requested are available to you, please do one of the following: Make a payment [1] of $35 or the same amount in your currency from the Transaction History page [2] and reply to this message when the charge clears. Your payment will be applied to any charges you incur in the future and will be visible as a credit in your account. Alternatively, reply to this message with the project ID of another project that you own that has cleared a charge of at least the amount mentioned. To learn more about project quota requests and cores quota requests, check out the Project Quota Request FAQ page at [3] and the Request cores quota increase FAQs page at [4]. We appreciate your patience. Sincerely, Cloud Platform Support [1] - https://support.google.com/cloud/answer/6294016 [2] - https://console.cloud.google.com/billing [3] - https://support.google.com/cloud/answer/6330231 [4] - https://support.google.com/cloud/answer/6376374
  11.  Hello, Your quota request for Project '632906615582' has been

    approved and your project quota has been adjusted accordingly. Changed Quota: +--------------------+-----------------+ | Region: asia-east1 | NVIDIA_K80_GPUS | +--------------------+-----------------+ | Changes | 0 -> 1 | +--------------------+-----------------+ To verify the quota change, please navigate to https://console.developers.google.com/project/632906615582/compute/quotas and verify what are your current project quotas. Happy Computing! Google for Work Support http://support.google.com/enterprisehelp/
  12. OWJEJBTNJ   ]/7*%*"4.*%SJWFS7FSTJPO] ]    ](16/BNF1FSTJTUFODF.]#VT*E%JTQ"]7PMBUJMF6ODPSS&$$] ]'BO5FNQ1FSG1XS6TBHF$BQ].FNPSZ6TBHF](166UJM$PNQVUF.]

    ]  ] ](F'PSDF(590GG]0O]/"] ]$188].J#.J#]%FGBVMU]        ]1SPDFTTFT(16.FNPSZ] ](161*%5ZQF1SPDFTTOBNF6TBHF] ]] ](VTSMJCYPSH9PSH.J#] ]$QZUIPO.J#]   
  13. I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_26: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_25: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841]

    save/RestoreV2_25: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_24: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_24: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_23: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_23: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_22: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_22: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_21: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_21: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_20: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 
  14. I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_8: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_7: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841]

    save/RestoreV2_7: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_6: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_6: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_5: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_5: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_4: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_4: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_3: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_3: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_2: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 
  15. I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2_2: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2_1: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841]

    save/RestoreV2_1: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 save/RestoreV2: (RestoreV2): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/RestoreV2: (RestoreV2)/ job:localhost/replica:0/task:0/cpu:0 conv3/biases/ExponentialMovingAverage: (VariableV2): /job:localhost/replica:0/task:0/ cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] conv3/biases/ ExponentialMovingAverage: (VariableV2)/job:localhost/replica:0/task:0/cpu:0 save/Assign_22: (Assign): /job:localhost/replica:0/task:0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] save/Assign_22: (Assign)/ job:localhost/replica:0/task:0/cpu:0 conv3/biases/ExponentialMovingAverage/read: (Identity): /job:localhost/replica:0/task: 0/cpu:0 I tensorflow/core/common_runtime/simple_placer.cc:841] conv3/biases/  ҎԼུ
  16. I tensorflow/core/common_runtime/simple_placer.cc:841] tower_0/image_loader_vvv/ ReaderReadV2: (ReaderReadV2)/job:localhost/replica:0/task:0/cpu:0 tower_0/image_loader_vvv/DecodeJpeg: (DecodeJpeg): /job:localhost/replica:0/task: 0/cpu:0 I

    tensorflow/core/common_runtime/simple_placer.cc:841] tower_0/image_loader_vvv/ DecodeJpeg: (DecodeJpeg)/job:localhost/replica:0/task:0/cpu:0 tower_0/image_loader_vvv/random_crop/Shape: (Shape): /job:localhost/replica:0/ task:0/gpu:0  EFDPEF@KQFH
  17. 

  18. 

  19. 

  20. 

  21. I tensorflow/core/common_runtime/simple_placer.cc:841] tower_0/image_loader_vvv/ ReaderReadV2: (ReaderReadV2)/job:localhost/replica:0/task:0/cpu:0 tower_0/image_loader_vvv/DecodeJpeg: (DecodeJpeg): /job:localhost/replica:0/task: 0/cpu:0 I

    tensorflow/core/common_runtime/simple_placer.cc:841] tower_0/image_loader_vvv/ DecodeJpeg: (DecodeJpeg)/job:localhost/replica:0/task:0/cpu:0 tower_0/image_loader_vvv/random_crop/Shape: (Shape): /job:localhost/replica:0/ task:0/gpu:0  EFDPEF@KQFH
  22. BMMPX@TPGU@QMBDFNFOU  If you would like TensorFlow to automatically choose

    an existing and supported device to run the operations in case the specified one doesn't exist, you can set allow_soft_placement to True in the configuration option when creating the session. https://www.tensorflow.org/tutorials/using_gpu
  23. EFG@MPBE@JNBHF GJMF@MJTU JOQVU@TJ[F PVUQVU@TJ[F DIBOOFMT TDBMF 
 XJUIUGOBNF@TDPQF JNBHF@MPBEFS@WWW 


    GJMFOBNF@RVFVFUGUSBJOTUSJOH@JOQVU@QSPEVDFS GJMF@MJTU 
 TIVGGMF5SVF 
 SFBEFSUG8IPMF'JMF3FBEFS 
 @ WBMVFSFBEFSSFBE GJMFOBNF@RVFVF 
 
 JNBHFUGJNBHFEFDPEF@KQFH WBMVF DIBOOFMTDIBOOFMT 
 DSPQQFE@JNBHFUGSBOEPN@DSPQ JNBHF  JOQVU@TJ[F JOQVU@TJ[F DIBOOFMT 
 
 MS@JNBHFUGJNBHFSFTJ[F@JNBHFT DSPQQFE@JNBHF  JOQVU@TJ[FTDBMF 
 JOQVU@TJ[FTDBMF 
 NFUIPEUGJNBHF3FTJ[F.FUIPE#*$6#*$ 
 
 MS@JNBHFUGJNBHFSFTJ[F@JNBHFT MS@JNBHF  JOQVU@TJ[F JOQVU@TJ[F 
 NFUIPEUGJNBHF3FTJ[F.FUIPE#*$6#*$ 
 
 PGGTFU@MFGU JOQVU@TJ[FPVUQVU@TJ[F 
 PGGTFU@UPQ JOQVU@TJ[FPVUQVU@TJ[F 
 HSPVOE@USVUIUGJNBHFDSPQ@UP@CPVOEJOH@CPY DSPQQFE@JNBHF 
 PGGTFU@UPQ PGGTFU@MFGU 
 PVUQVU@TJ[F PVUQVU@TJ[F 
 
 SFUVSOMS@JNBHF HSPVOE@USVUI 
  24. EFG@MPBE@JNBHF GJMF@MJTU JOQVU@TJ[F PVUQVU@TJ[F DIBOOFMT TDBMF CBUDI@TJ[F 
 XJUIUGOBNF@TDPQF JNBHF@MPBEFS@WWW

    
 GJMFOBNF@RVFVFUGUSBJOTUSJOH@JOQVU@QSPEVDFS GJMF@MJTU 
 TIVGGMF5SVF 
 SFBEFSUG8IPMF'JMF3FBEFS 
 @ WBMVFSFBEFSSFBE GJMFOBNF@RVFVF 
 JNBHFUGJNBHFEFDPEF@KQFH WBMVF DIBOOFMTDIBOOFMT 
 
  25. MS@JNBHFT<>
 HSPVOE@USVUIT<>
 
 GPSJJOSBOHF CBUDI@TJ[F 
 DSPQQFE@JNBHFUGSBOEPN@DSPQ JNBHF  JOQVU@TJ[F

    JOQVU@TJ[F DIBOOFMT 
 MS@JNBHFUGJNBHFSFTJ[F@JNBHFT DSPQQFE@JNBHF  JOQVU@TJ[FTDBMF 
 JOQVU@TJ[FTDBMF 
 NFUIPEUGJNBHF3FTJ[F.FUIPE#*$6#*$ 
 MS@JNBHFUGJNBHFSFTJ[F@JNBHFT MS@JNBHF  JOQVU@TJ[F JOQVU@TJ[F 
 NFUIPEUGJNBHF3FTJ[F.FUIPE#*$6#*$ 
 MS@JNBHFUGFYQBOE@EJNT MS@JNBHF EJN 
 MS@JNBHFTBQQFOE MS@JNBHF 
 
 PGGTFU@MFGU JOQVU@TJ[FPVUQVU@TJ[F 
 PGGTFU@UPQ JOQVU@TJ[FPVUQVU@TJ[F 
 HSPVOE@USVUIUGJNBHFDSPQ@UP@CPVOEJOH@CPY DSPQQFE@JNBHF 
 PGGTFU@UPQ PGGTFU@MFGU 
 PVUQVU@TJ[F PVUQVU@TJ[F 
 HSPVOE@USVUIUGFYQBOE@EJNT HSPVOE@USVUI EJN 
 HSPVOE@USVUITBQQFOE HSPVOE@USVUI 
 
 MS@JNBHFTUGDPODBU MS@JNBHFT BYJT 
 HSPVOE@USVUITUGDPODBU HSPVOE@USVUIT BYJT 
 
 SFUVSOMS@JNBHFT HSPVOE@USVUIT
 
  26. MS@JNBHF@CBUDI HSPVOE@USVUI@CBUDIUGUSBJOTIVGGMF@CBUDI 
 <JNBHFT HSPVOE@USVUIT> 
 CBUDI@TJ[F'-"(4CBUDI@TJ[F 
 DBQBDJUZDBQBDJUZ 


    FORVFVF@NBOZ5SVF 
 NJO@BGUFS@EFRVFVF'-"(4NJO@BGUFS@EFRVFVF 
 OVN@UISFBET'-"(4OVN@UISFBET 
  UGUSBJOTIV⒐F@CBUDI
  27.  4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ

    4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 4UFQ -PTT 5JNFNTTUFQ 128 sample / batch learning rate 0.001
  28. Impress R&DࣾΑΓ
 Print On Demand ˍ ిࢠॻ੶Ͱ ݄̑Լ० ൃചܾఆ ݄̐̕೔ʢ೔ʣ

    ΞΩόɾεΫΤΞ ʰٕज़ॻయ̎ ͓ʔ̍̎ʱʹͯ TensorFlow͸͡Ί·ͨ͠
 Super Resolution ʔ ௒ղ૾ ׬ച͠·ͨ͠