Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SSII2024 [OS2] 画像、その先へ 〜モーション解析への誘い〜

SSII2024 [OS2] 画像、その先へ 〜モーション解析への誘い〜

More Decks by 画像センシングシンポジウム

Other Decks in Research

Transcript

  1. ʜͱ͍͏ͷ΋ ͜Ε·Ͱ͋·Γը૾ͱ޲͖߹ͬͯ͜ͳ͔ͬͨ fixed weights trainable weights !! " !# "

    !! !# ""(! # ) ""(! ! ) ""(!$ ) !$ !$ " !! " !# " !$ " !! !# !$ GT NN SLR GSLR (ours) <'VKJXBSB )BTIJNPUP$713> <-J *$$7> <-J *$$7>
  2.  ಈ࡞ͷऔಘํ๏ ϞʔγϣϯΩϟϓνϟʔσόΠε • ޫֶࣜ • ׳ੑࣜ • ࣓ؾࣜ ಈը͔Βਪఆ͢Δݚڀ΋׆ൃ

    ՄࢹԽIUUQTMPUIHJUIVCJPPMZNQF#7)@QMBZFSIUNM ޫֶࣜ<$.6.PDBQ%BUBCBTF IUUQNPDBQDTDNVFEV>
  3.  ಈ࡞ͷදݱํ๏ ֶश༻ʹ౷Ұදݱʹม׵͢Δඞཁੑ • ,*5εέϧτϯ<1MBQQFSU > • 4.1-εέϧτϯ<-PQFS > •

    ࠊɺؔઅͷؔઅճసɾ଎౓౳ΛؚΉߴ࣍ݩදݱʹม׵͢Δ͜ͱ΋ • σʔλϑϨʔϜ਺Y ؔઅ਺Yදݱ࣍ݩ 0 1 2 3 5 4 6 9 8 7 11 10 14 13 12 15 17 16 18 19 20 21 0 11 2 5 8 Frame i Frame i+1 Frame i+(N-1) … … SMPL Skeleton Motion Sequence of Right Leg Extract Interpolate 0 1 2 3 4 8 9 10 5 6 7 16 17 18 19 20 11 12 13 14 15 Frame i Frame i+1 Frame i+(N-1) … … … N Frames N Points Motion Patch of Right Leg Motion Sequence of Right Leg Interpolate Extract KIT Skeleton 0 14 11 12 13 15
  4. 7)-ͷݚڀ׆ಈ  :V 'VKJXBSB l'SBNF-FWFM-BCFM3FGJOFNFOUGPS4LFMFUPO#BTFE8FBLMZ4VQFSWJTFE5FNQPSBM "DUJPO-PDBMJ[BUJPOz """* :V 5BOBLB 'VKJXBSB

    l&YQMPSJOH7JTJPO5SBOTGPSNFSTGPS%)VNBO.PUJPO-BOHVBHF.PEFMT XJUI.PUJPO1BUDIFT z$713 UPBQQFBS 5BOBLB 'VKJXBSB l3PMFBXBSF*OUFSBDUJPO(FOFSBUJPOGSPN5FYUVBM%FTDSJQUJPO z*$$7 
  5.  ՝୊̎ σʔλ਺ ໰σʔλ਺ͷগͳ͞Λ৐Γӽ͑Δํ๏͸ͳ͍͔ʁ ˠϞʔγϣϯͷը૾ԽΛఏএ 0 1 2 3 5

    4 6 9 8 7 11 10 14 13 12 15 17 16 18 19 20 21 0 11 2 5 8 Frame i Frame i+1 Frame i+(N-1) … … … N Frames N Points Motion Patch of Right Leg SMPL Skeleton Motion Sequence of Right Leg Extract Interpolate
  6.  .PUJPO1BUDIFT ֤෦ҐΛ7J5ͷύονͱͯ͠ѻ͑Δ.PUJPO1BUDI<:V $713> ˠಈ͖Λ৭ͷάϥσʔγϣϯͰදݱ ܇࿅ࡁΈ7J5͕ར༻Մೳʹ   #Body Parts

    #Frames Right Leg Left Leg Torso Right Arm Left Arm a person brings their arms down Motion Patches #Body Parts #Frames a person takes a small step forward Motion Patches Right Leg Left Leg Torso Right Arm Left Arm #Fram es Transformer Motion Features … #Body Parts #Frames #Fram es Vision Transformer Motion Features … #Frames #Joints Existing Methods Proposed Method … … … Motion Patches (Shared by skeleton) Skeleton Sequence (Differed by skeleton) Pre-trained weights for 2D images available N Frames N Points
  7.  ݁Ռ ը૾υϝΠϯ͔Βͷ஌ࣝΛར༻͠ਫ਼౓޲্ୡ੒ ҟͳΔεέϧτϯߏ଄ͷσʔλΛNPUJPOQBUDIʹม׵͠ɺసҠֶश΋Մೳ Rank #1 Rank #2 Rank #1

    Person runs in a zigzag motion and ducks under an invisible object halfway through then returns to full height. Person walks six steps to side with right arm up. A person walks slowly towards the right while slowly raising a hand and then walking backwards. The man moves to his right. Person walks six steps to side with right arm up. Rank #2 TMR Ours A person who seems to evade something from their left side and run at a insane pace. A person strafes to the right. Person runs in a zigzag motion and ducks under an invisible object halfway through then returns to full height. A man crouches down while quickly walking forward and then stands up straight. A person walking and helping maintain their balance and support, from holding onto a side rail or wall. A person runs diagonally across a room with their arms swinging hands down. A person who seems to evade something from their left side and run at a insane pace. A person running then quickly taking a step to their right. A person runs forward with one leg crossing in front of the other repetitively before coming to a stop. A person runs diagonally across a room with their arms swinging hands down.
  8.  ໾ׂʹ߹ͬͨจ΁l຋༁z ೋਓͷߦಈͷࡍɺͦΕͧΕʹ໾ׂ͕͋Δ͜ͱΛఏএ<5BOBLB *$$7> • ରশతަྲྀˠςΩετ͸ͦͷ·· • ඇରশަྲྀˠೳಈɾडಈͷจষʹ຋༁ɺରԠ͢Δߦಈʹ෇༩ֶ͠श 5SBOTGPSNFS QBSBNFUFSTIBSF

    5SBOTGPSNFS QBSBNFUFSTIBSF "DUJWFWPJDF "QFSTPOJT QVTIJOH OPJTF $SPTT"UUFOUJPO 1BTTJWFWPJDF "QFSTPOJT QVTIFECZ OPJTF -BOHVBHF &ODPEFS -BOHVBHF &ODPEFS "DUPS 3FDFJWFS
  9.  ݁Ռ ໾ׂͷจষΛ໌֬Խ͢Δ͜ͱʹΑΓɺੜ੒ਫ਼౓͕޲্ ΞςϯγϣϯػߏͷಋೖʹΑΓ௰᧒ͷ͋͏ಈ࡞Λ࣮ݱ 0VST 5XP5SBOTGPSNFS "DUPS "DUJWF7PJDF  "QFSTPOJTLJDLJOH

    UIFPUIFSQFSTPO 3FDFJWFS 1BTTJWF7PJDF  "QFSTPOJTLJDLFECZ UIFPUIFSQFSTPO "DUPS "DUJWF7PJDF  "QFSTPOJTQVTIJOH UIFPUIFSQFSTPO 3FDFJWFS 1BTTJWF7PJDF  "QFSTPOJTQVTIFECZ UIFPUIFSQFSTPO આ໌จ
  10.  ؔ࿈จݙ • 1MBQQFSU .BUUIJBT $ISJTUJBO.BOEFSZ BOE5BNJN"TGPVS5IFLJUNPUJPOMBOHVBHFEBUBTFU #JH EBUB 

    OP   • -PQFS .BUUIFX /BVSFFO .BINPPE +BWJFS3PNFSP (FSBSE1POT.PMM BOE.JDIBFM+#MBDL4.1- "TLJOOFENVMUJQFSTPOMJOFBSNPEFM*O 4FNJOBM(SBQIJDT1BQFST1VTIJOHUIF#PVOEBSJFT 7PMVNF  QQ • 1FUSPWJDI .BUIJT .JDIBFM+#MBDL BOE(ÛM7BSPM5.35FYUUPNPUJPOSFUSJFWBMVTJOHDPOUSBTUJWF %IVNBONPUJPOTZOUIFTJT*O 1SPDFFEJOHTPGUIF*&&&$7'*OUFSOBUJPOBM$POGFSFODFPO$PNQVUFS 7JTJPO QQ • ;IBOH +JBOSPOH :BOHTPOH ;IBOH 9JBPEPOH $VO 4IBPMJ )VBOH :POH;IBOH )POHXFJ;IBP  )POHUBP -V BOE9J4IFO5NHQU(FOFSBUJOHIVNBONPUJPOGSPNUFYUVBMEFTDSJQUJPOTXJUI EJTDSFUFSFQSFTFOUBUJPOT BS9JW QSFQSJOUBS9JW   • 5BOBLB %BJLJ %BJLJ*LBNJ 5PTIJIJLP:BNBTBLJ BOE,JZPIBSV "J[BXB+PJOUPQUJNJ[BUJPOGSBNFXPSL GPSMFBSOJOHXJUIOPJTZMBCFMT*O 1SPDFFEJOHTPGUIF*&&&DPOGFSFODFPODPNQVUFSWJTJPOBOEQBUUFSO SFDPHOJUJPO QQ • $.6(SBQIJDT-BC.PUJPO$BQUVSF%BUBCBTF IUUQNPDBQDTDNVFEV