Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Leveraging Temporal Joint Depths for Improving ...

Naoki Kato
November 04, 2020

Leveraging Temporal Joint Depths for Improving 3D Human Pose Estimation in Video

Naoki Kato, Hiroto Honda, Yusuke Uchida, "Leveraging Temporal Joint Depths for Improving 3D Human Pose Estimation in Video," In GCCE 2020.

https://arxiv.org/abs/2011.02172

Naoki Kato

November 04, 2020
Tweet

More Decks by Naoki Kato

Other Decks in Research

Transcript

  1. !]DQYQjs0IEP[]Y]OQIh ] jG 'KXPDQSRVHHVWLPDWLRQ Ŷ /RFDOL]HKXPDQMRLQWVLQD'FRRUGLQDWHV\VWHP Ŷ $SSOLFDWLRQV Ŷ 0RWLRQFDSWXUHV\VWHP

    Ŷ $595 Ŷ +XPDQDFWLRQDQDO\VLV Ŷ ,QYHKLFOHFDPHUDYLGHRDQDO\VLV Ŷ 'LIILFXOWLHV Ŷ +DUGWREXLOGKLJKO\GLYHUVHGDWDVHW Ŷ $PELJXLW\LQMRLQWGHSWK %DFNJURXQG
  2. !]DQYQjs0IEP[]Y]OQIh ] jG 8VLQJWKHFHQWURLGRIWKHKHDWPDSDVWKHHVWLPDWHGMRLQWFRRUGLQDWH Ŷ (QGWRHQGLQIHUHQFHRQMRLQWFRRUGLQDWHVXVLQJKHDWPDSV Ŷ &DQEHWUDLQHGXVLQJERWK'DQG'GDWDVHWV )LUVW6WDJH,QWHJUDO5HJUHVVLRQ>6XQ(&&9¶@ Ŷ

    $UFKLWHFWXUH Ŷ %DFNERQH5HV1HW Ŷ +HDGGHFRQYROXWLRQOD\HUVWRSURGXFH'KHDWPDSV ;6XQ%;LDR):HL6/LDQJ<:HL³,QWHJUDO+XPDQ3RVH5HJUHVVLRQ´,Q(&&9
  3. !]DQYQjs0IEP[]Y]OQIh ] jG (VWLPDWH'SRVHLQWKHFHQWHUIUDPHIURPDQLQSXWVHTXHQFHXVLQJ'&RQY1HW ZLWKWHPSRUDOGLODWHGFRQYROXWLRQV Ŷ 6WDFNRIUHVLGXDOEORFNV FRQY%15H/8GURSRXWî Ŷ $SSO\WHPSRUDOGLODWHGFRQYROXWLRQRQWKHLQSXWOD\HUDQGWKHILUVWOD\HURIHDFKUHVLGXDOEORFN

    Ŷ 7HPSRUDOUHFHSWLYHILHOGV:% :NHUQHOVL]H%WKHQXPEHURIUHVLGXDOEORFNV Ŷ ([WHQGLQSXWGLPHQVLRQVIRUGHSWK>WHPSRUDOOHQJWK@î>WKHQXPEHURIMRLQWV@î 6HFRQG6WDJH'&RQY1HW>3DYOOR&935¶@ '3DYOOR&)HLFKWHQKRIHU'*UDQJLHU0$XOL³'KXPDQSRVHHVWLPDWLRQLQYLGHRZLWKWHPSRUDO FRQYROXWLRQVDQGVHPLVXSHUYLVHGWUDLQLQJ´,Q&935
  4. !]DQYQjs0IEP[]Y]OQIh ] jG 2XUDSSURFK Ŷ )LUVWVWDJH Ŷ ,QSXWLPDJH Ŷ 2XWSXW'SRVH

    'SRVHGHSWK Ŷ 6HFRQGVWDJH Ŷ ,QSXW'SRVHGHSWKVHTXHQFH Ŷ 2XWSXW'SRVH ([LVWLQJDSSURDFK Ŷ )LUVWVWDJH Ŷ ,QSXWLPDJH Ŷ 2XWSXW'SRVH Ŷ 6HFRQGVWDJH Ŷ ,QSXW'SRVHVHTXHQFH Ŷ 2XWSXW'SRVH &RPSDULVRQRIRXUDSSURDFKZLWKH[LVWLQJDSSURDFKHV 0RGHO6XPPDU\ —5HGXFHGGHSWKDPELJXLW\LQWKHVHFRQGVWDJH 7KHGHSWKSUHGLFWHGE\WKHILUVWVWDJHPLJKWEHHUURQHRXV
  5. !]DQYQjs0IEP[]Y]OQIh ] jG Ŷ )LUVWVWDJH 'DWDDXJPHQWDWLRQ Ŷ +RUL]RQWDOIOLSRQSRVH Ŷ 6FDOLQJ

    Ŷ $GGJDXVVLDQQRLVHWR'SRVH Ŷ $GGJDXVVLDQQRLVHWRMRLQWGHSWK 'DWDDXJPHQWDWLRQ Ŷ +RUL]RQWDOIOLSRQLPDJH Ŷ 5RWDWLRQ Ŷ 6FDOLQJ Ŷ %%R[6KLIWLQJ Ŷ &RORUMLWWHULQJ 7UDLQLQJGHWDLOV Ŷ 2SWLPL]HU$GDP: ZHLJKWGHFD\H Ŷ %DWFKVL]H Ŷ 'HFD\OHDUQLQJUDWHIURPHWRHZLWK FRVLQHIXQFWLRQIRUHSRFKV 7UDLQLQJGHWDLOV Ŷ 6L]HRILQSXWLPDJHî Ŷ 2SWLPL]HU$GDP Ŷ %DWFKVL]H Ŷ 'HFD\OHDUQLQJUDWHIURPHWRHZLWK FRVLQHIXQFWLRQIRUHSRFKV ,PSOHPHQWDWLRQGHWDLOV Ŷ 6HFRQGVWDJH
  6. !]DQYQjs0IEP[]Y]OQIh ] jG Ŷ 'DWDVHW+XPDQ0 Ŷ VHTXHQFHV Ŷ VXEMHFWV IRUWUDLQIRUYDOLGDWLRQ

    Ŷ FDPHUDV Ŷ DFWLYLWLHVîSDWWHUQV Ŷ (YDOXDWLRQPHWULFV Ŷ 0HDQ3HU-RLQW3RVLWLRQ(UURU 03-3(  $YHUDJHGGLVWDQFHVEHWZHHQWKHSUHGLFWHGDQG JURXQGWUXWKFRRUGLQDWHV Ŷ 3$03-3( 03-3(DIWHUULJLGDOLJQPHQW ([SHULPHQWDO6HWXS 'DWDH[DPSOHRI+XPDQ0
  7. !]DQYQjs0IEP[]Y]OQIh ] jG 0RGHOSHUIRUPDQFHZLWKGLIIHUHQWLQSXWVDQGWHPSRUDOUHFHSWLYHILHOGV 7KHUHVSHFWLYHPRGHOVKDYH NHUQHOVL]HUHVLGXDOEORFNV RI   

      DQG  Ŷ ,QSXWWLQJMRLQWGHSWKVGLUHFWO\LQWUDLQLQJLQFUHDVHVWKHHUURU Ŷ $GGLQJJDXVVLDQQRLVHWRGHSWKVLQWUDLQLQJLQFUHDVHVWKHPRGHOSHUIRUPDQFH Ŷ ,QFUHDVHGSHUIRUPDQFHZLWKODUJHUHFHSWLYHILHOG 03-3(RIWKHILUVWVWDJHLVPP 6IWYPXW ,QSXW 5HFHSWLYHILHOG     'SRVH     'SRVHGHSWK ZRQRLVH     'SRVHGHSWK QRLVHı      03-3( PP RIHDFKPRGHO
  8. !]DQYQjs0IEP[]Y]OQIh ] jG 'SQTEVMWSR[MXLXLI)\MWXMRK1IXLSHW ᡭἲ 03-3( PP 3$03-3( PP 0DUWLQH]HWDO

    ,&&9¶   6XQHWDO (&&9¶   &DLHWDO ,&&9¶ ‚   3DYOORHWDO &935¶ ‚   2XUV‚   &RPSDULVRQZLWKWKHH[LVWLQJPHWKRGV 䠄‚H[SORLWVWHPSRUDOLQIRUPDWLRQ䠅 Ŷ $FKLHYHWKHKLJKHVWSHUIRUPDQFHLQERWKPHWULFV Ŷ PPORZHU03-3(DQGPPORZHU3$03-3(WKDQWKHPHWKRGRI3DYOORHWDO
  9. !]DQYQjs0IEP[]Y]OQIh ] jG Ŷ &RQFOXVLRQ Ŷ 3URSRVHWRXVHMRLQWGHSWKVDVDQLQWHUPHGLDWHUHSUHVHQWDWLRQIRUHVWLPDWLQJ'KXPDQ SRVHIURPYLGHRV Ŷ 3URSRVHWRDGGJDXVVLDQQRLVHWRWKHLQSXWVRIWKHVHFRQGVWDJHWRPLPLFWKHWHVWWLPH

    SUHGLFWLRQVRIWKHILUVWVWDJH Ŷ 'HPRQVWUDWHHIIHFWLYHQHVVRIWKHSURSRVHGDSSURDFKRQ+XPDQ0 Ŷ )XWXUH:RUNV Ŷ $PRUHHIIHFWLYHGDWDDXJPHQWDWLRQPHWKRGIRUWUDLQLQJWZRVWDJHPRGHO Ŷ $SLSHOLQHWRH[SOLFLWO\GHDOZLWKRFFOXGHGIUDPHV &RQFOXVLRQDQG)XWXUH:RUNV