Share
Embed
Leveraging Temporal Joint Depths for Improving 3D Human Pose Estimation in Video
Naoki Kato, Hiroto Honda, Yusuke Uchida, "Leveraging Temporal Joint Depths for Improving 3D Human Pose Estimation in Video," In GCCE 2020.
https://arxiv.org/abs/2011.02172
Naoki Kato
November 04, 2020
Transcript
16 slides
-
!]DQYQjs0IEP[]Y]OQIh ] jG 'KXPDQSRVHHVWLPDWLRQ Ŷ /RFDOL]HKXPDQMRLQWVLQD'FRRUGLQDWHV\VWHP Ŷ $SSOLFDWLRQV Ŷ 0RWLRQFDSWXUHV\VWHP
Ŷ $595 Ŷ +XPDQDFWLRQDQDO\VLV Ŷ ,QYHKLFOHFDPHUDYLGHRDQDO\VLV Ŷ 'LIILFXOWLHV Ŷ +DUGWREXLOGKLJKO\GLYHUVHGDWDVHW Ŷ $PELJXLW\LQMRLQWGHSWK %DFNJURXQG -
!]DQYQjs0IEP[]Y]OQIh ] jG 8VLQJWKHFHQWURLGRIWKHKHDWPDSDVWKHHVWLPDWHGMRLQWFRRUGLQDWH Ŷ (QGWRHQGLQIHUHQFHRQMRLQWFRRUGLQDWHVXVLQJKHDWPDSV Ŷ &DQEHWUDLQHGXVLQJERWK'DQG'GDWDVHWV )LUVW6WDJH,QWHJUDO5HJUHVVLRQ>6XQ(&&9¶@ Ŷ
$UFKLWHFWXUH Ŷ %DFNERQH5HV1HW Ŷ +HDGGHFRQYROXWLRQOD\HUVWRSURGXFH'KHDWPDSV ;6XQ%;LDR):HL6/LDQJ<:HL³,QWHJUDO+XPDQ3RVH5HJUHVVLRQ´,Q(&&9 -
!]DQYQjs0IEP[]Y]OQIh ] jG (VWLPDWH'SRVHLQWKHFHQWHUIUDPHIURPDQLQSXWVHTXHQFHXVLQJ'&RQY1HW ZLWKWHPSRUDOGLODWHGFRQYROXWLRQV Ŷ 6WDFNRIUHVLGXDOEORFNV FRQY%15H/8GURSRXWî Ŷ $SSO\WHPSRUDOGLODWHGFRQYROXWLRQRQWKHLQSXWOD\HUDQGWKHILUVWOD\HURIHDFKUHVLGXDOEORFN
Ŷ 7HPSRUDOUHFHSWLYHILHOGV:% :NHUQHOVL]H%WKHQXPEHURIUHVLGXDOEORFNV Ŷ ([WHQGLQSXWGLPHQVLRQVIRUGHSWK>WHPSRUDOOHQJWK@î>WKHQXPEHURIMRLQWV@î 6HFRQG6WDJH'&RQY1HW>3DYOOR&935¶@ '3DYOOR&)HLFKWHQKRIHU'*UDQJLHU0$XOL³'KXPDQSRVHHVWLPDWLRQLQYLGHRZLWKWHPSRUDO FRQYROXWLRQVDQGVHPLVXSHUYLVHGWUDLQLQJ´,Q&935 -
!]DQYQjs0IEP[]Y]OQIh ] jG 2XUDSSURFK Ŷ )LUVWVWDJH Ŷ ,QSXWLPDJH Ŷ 2XWSXW'SRVH
'SRVHGHSWK Ŷ 6HFRQGVWDJH Ŷ ,QSXW'SRVHGHSWKVHTXHQFH Ŷ 2XWSXW'SRVH ([LVWLQJDSSURDFK Ŷ )LUVWVWDJH Ŷ ,QSXWLPDJH Ŷ 2XWSXW'SRVH Ŷ 6HFRQGVWDJH Ŷ ,QSXW'SRVHVHTXHQFH Ŷ 2XWSXW'SRVH &RPSDULVRQRIRXUDSSURDFKZLWKH[LVWLQJDSSURDFKHV 0RGHO6XPPDU\ 5HGXFHGGHSWKDPELJXLW\LQWKHVHFRQGVWDJH 7KHGHSWKSUHGLFWHGE\WKHILUVWVWDJHPLJKWEHHUURQHRXV -
!]DQYQjs0IEP[]Y]OQIh ] jG Ŷ )LUVWVWDJH 'DWDDXJPHQWDWLRQ Ŷ +RUL]RQWDOIOLSRQSRVH Ŷ 6FDOLQJ
Ŷ $GGJDXVVLDQQRLVHWR'SRVH Ŷ $GGJDXVVLDQQRLVHWRMRLQWGHSWK 'DWDDXJPHQWDWLRQ Ŷ +RUL]RQWDOIOLSRQLPDJH Ŷ 5RWDWLRQ Ŷ 6FDOLQJ Ŷ %%R[6KLIWLQJ Ŷ &RORUMLWWHULQJ 7UDLQLQJGHWDLOV Ŷ 2SWLPL]HU$GDP: ZHLJKWGHFD\H Ŷ %DWFKVL]H Ŷ 'HFD\OHDUQLQJUDWHIURPHWRHZLWK FRVLQHIXQFWLRQIRUHSRFKV 7UDLQLQJGHWDLOV Ŷ 6L]HRILQSXWLPDJHî Ŷ 2SWLPL]HU$GDP Ŷ %DWFKVL]H Ŷ 'HFD\OHDUQLQJUDWHIURPHWRHZLWK FRVLQHIXQFWLRQIRUHSRFKV ,PSOHPHQWDWLRQGHWDLOV Ŷ 6HFRQGVWDJH -
!]DQYQjs0IEP[]Y]OQIh ] jG Ŷ 'DWDVHW+XPDQ0 Ŷ VHTXHQFHV Ŷ VXEMHFWV IRUWUDLQIRUYDOLGDWLRQ
Ŷ FDPHUDV Ŷ DFWLYLWLHVîSDWWHUQV Ŷ (YDOXDWLRQPHWULFV Ŷ 0HDQ3HU-RLQW3RVLWLRQ(UURU 03-3( $YHUDJHGGLVWDQFHVEHWZHHQWKHSUHGLFWHGDQG JURXQGWUXWKFRRUGLQDWHV Ŷ 3$03-3( 03-3(DIWHUULJLGDOLJQPHQW ([SHULPHQWDO6HWXS 'DWDH[DPSOHRI+XPDQ0 -
!]DQYQjs0IEP[]Y]OQIh ] jG 0RGHOSHUIRUPDQFHZLWKGLIIHUHQWLQSXWVDQGWHPSRUDOUHFHSWLYHILHOGV 7KHUHVSHFWLYHPRGHOVKDYH NHUQHOVL]HUHVLGXDOEORFNV RI
DQG Ŷ ,QSXWWLQJMRLQWGHSWKVGLUHFWO\LQWUDLQLQJLQFUHDVHVWKHHUURU Ŷ $GGLQJJDXVVLDQQRLVHWRGHSWKVLQWUDLQLQJLQFUHDVHVWKHPRGHOSHUIRUPDQFH Ŷ ,QFUHDVHGSHUIRUPDQFHZLWKODUJHUHFHSWLYHILHOG 03-3(RIWKHILUVWVWDJHLVPP 6IWYPXW ,QSXW 5HFHSWLYHILHOG 'SRVH 'SRVHGHSWK ZRQRLVH 'SRVHGHSWK QRLVHı 03-3( PP RIHDFKPRGHO -
!]DQYQjs0IEP[]Y]OQIh ] jG 'SQTEVMWSR[MXLXLI)\MWXMRK1IXLSHW ᡭἲ 03-3( PP 3$03-3( PP 0DUWLQH]HWDO
,&&9¶ 6XQHWDO (&&9¶ &DLHWDO ,&&9¶ 3DYOORHWDO &935¶ 2XUV &RPSDULVRQZLWKWKHH[LVWLQJPHWKRGV 䠄H[SORLWVWHPSRUDOLQIRUPDWLRQ䠅 Ŷ $FKLHYHWKHKLJKHVWSHUIRUPDQFHLQERWKPHWULFV Ŷ PPORZHU03-3(DQGPPORZHU3$03-3(WKDQWKHPHWKRGRI3DYOORHWDO -
!]DQYQjs0IEP[]Y]OQIh ] jG Ŷ &RQFOXVLRQ Ŷ 3URSRVHWRXVHMRLQWGHSWKVDVDQLQWHUPHGLDWHUHSUHVHQWDWLRQIRUHVWLPDWLQJ'KXPDQ SRVHIURPYLGHRV Ŷ 3URSRVHWRDGGJDXVVLDQQRLVHWRWKHLQSXWVRIWKHVHFRQGVWDJHWRPLPLFWKHWHVWWLPH
SUHGLFWLRQVRIWKHILUVWVWDJH Ŷ 'HPRQVWUDWHHIIHFWLYHQHVVRIWKHSURSRVHGDSSURDFKRQ+XPDQ0 Ŷ )XWXUH:RUNV Ŷ $PRUHHIIHFWLYHGDWDDXJPHQWDWLRQPHWKRGIRUWUDLQLQJWZRVWDJHPRGHO Ŷ $SLSHOLQHWRH[SOLFLWO\GHDOZLWKRFFOXGHGIUDPHV &RQFOXVLRQDQG)XWXUH:RUNV