Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
【CVPR 2024】VTimeLLM: Empower LLM to Grasp Video...
Search
mori yuichiro
July 09, 2024
60
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
【CVPR 2024】VTimeLLM: Empower LLM to Grasp Video Moments 論文読み会発表資料
mori yuichiro
July 09, 2024
More Decks by mori yuichiro
See All by mori yuichiro
【EMNLP 2023】Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs 論文読み会発表資料
zaemon1251hesty
0
120
Featured
See All Featured
Google's AI Overviews - The New Search
badams
0
1k
Leveraging LLMs for student feedback in introductory data science courses - posit::conf(2025)
minecr
1
290
Between Models and Reality
mayunak
4
340
Money Talks: Using Revenue to Get Sh*t Done
nikkihalliwell
0
250
Visualization
eitanlees
152
17k
Building an army of robots
kneath
306
46k
The Art of Programming - Codeland 2020
erikaheidi
57
14k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
3.4k
Unlocking the hidden potential of vector embeddings in international SEO
frankvandijk
0
840
Being A Developer After 40
akosma
91
590k
Lightning talk: Run Django tests with GitHub Actions
sabderemane
0
200
Faster Mobile Websites
deanohume
310
31k
Transcript
75JNF--.&NQPXFS--.UP (SBTQ7JEFP.PNFOUT #JO)VBOH 9JO8BOH )POH$IFO ;JIBO4POH 8FOXV;IV 5TJOHIVB6OJWFSTJUZ $713 :VJDIJSP.PSJ
1BQFS3FBEJOH 1
4VNNBSZ • -BSHFMBOHVBHFNPEFMT --.T IBWFCFFOBEBQUFEUPIBOEMFWJEFP EBUBCVUTUSVHHMFXJUIQSFDJTFFWFOUUJNJOH • 75JNF--. BOFX7JEFP--. BEESFTTFTUIJTCZVTJOHBUISFF
TUBHFUSBJOJOHTUSBUFHZUPJNQSPWFUFNQPSBMVOEFSTUBOEJOH • &YQFSJNFOUTTIPX75JNF--.FYDFMTJOGJOFHSBJOFEWJEFP DPNQSFIFOTJPOBOEPVUQFSGPSNTFYJTUJOH7JEFP--.TJOWBSJPVT UBTLT 2
#BDLHSPVOE • 7JEFP--. • VUJMJ[JOH-BSHF-BOHVBHF.PEFMUPVOEFSTUBOEWJEFP • BOBHFOUGPSJOUFSBDUJOHXJUIIVNBOCBTFEPOBWJEFP • FH
7JEFP--B." <;IBOH &./-1> 7JEFP$IBU <-J BS9JW> 3 7JEFP--B."
1SPCMFN • &YJTUJOH7JEFP--.TGPDVTPOHFOFSBUJPOPGHFOFSJDWJEFP DBQUJPOT UIVTGBJMUPDBQUVSFUIFSFMBUJPOTIJQTCFUXFFOTQFDJGJD CPVOEFEFWFOUT 4 7JEFP$IBU 7JEFP--B."DBOOPU
MPDBMJ[FUIFFWFOU UJNJOH 7JEFP--.TBSF GBSGSPNQSFDJTFMZ VOEFSTUBOEJOH WJEFP
3FTFBSDI0CKFDUJWF 5 5IFZCVJME75JNF--.5IF'JSTUCPVOEBSZBXBSF7JEFP--. 5BDLMJOHUXPDIBMMFOHFTGPS7JEFP--.TUPDBQUVSFFWFOUCPVOEBSJFTJO WJEFP 4DBSDJUZPGMBSHFTDBMFCPVOEBSZBXBSFWJEFPEBUBTFUT Ø 5IFZDSFBUFEBUBTFUTJODPSQPSBUJOHFYJTUJOHUFNQPSBMSFMBUFE 7JEFP6OEFSTUBOEJOHUBTLT
%FTJHOFGGFDUJWFUFNQPSBMSFMBUFEWJEFPUBTLTGPSUSBJOJOH7JEFP--. Ø 5IFZQSPQPTF#PVOEBSZBXBSF5SBJOJOH4USBUFHZ
1SPQPTFE.FUIPE • 1SFMJNJOBSZ5XPUFNQPSBMSFMBUFE7JEFP6OEFSTUBOEJOHUBTLT 6 5FNQPSBM7JEFP (SPVOEJOH 57( %FOTF7JEFP $BQUJPOJOH %7$
• UFNQPSBMMPDBMJ[BUJPODPSSFTQPOEJOHWJEFP TFHNFOUTGPSHJWFOUFYUVBMJOQVUT • UFNQPSBMMPDBMJ[BUJPOBOEDBQUJPOJOH GPSBMMFWFOUT 5FNQPSBMMZ(SPVOEJOH/BUVSBM4FOUFODFJO7JEFP <+JOHZVBO &./-1> %FOTF$BQUJPOJOHJOFWFOUT <,SJTIOB *$$7>
1SPQPTFE.FUIPE • #PVOEBSZBXBSF5SBJOJOH4USBUFHZ DPOTJTUT5SBJOJOH4UBHFT 7
1SPQPTFE.FUIPE • 4UBHF'FBUVSF"MJHONFOU • "JNTUPUSBJOUIFWJTVBMBEBQUFS UPBMJHOWJEFP GFBUVSFTXJUI--.`TTFNBOUJDTQBDF • %BUBTFU-$4, JNBHFUFYUQBJSVTFEGPS
QSFUSBJOJOH--B7"<-JV /FVS*14`> • /PUF • 4JODFDPOUFNQPSBSZMBSHFTDBMFWJEFPUFYUEBUBTFUTDPOUBJO TVCTUBOUJBMUFYUVBMOPJTF UIFZVTFUIFJNBHFUFYUEBUBTFU 8
1SPQPTFE.FUIPE • 4UBHF#PVOEBSZ1FSDFQUJPO • "JNTUPGBDJMJUBUJOHUIFVOEFSTUBOEJOHPG WBSJPVTFWFOUTPDDVSSJOHXJUIJOUIFWJEFP • %BUBTFUTJOHMFUVSOBOENVMUJUVSO RVFTUJPOBOTXFSJOH 2"
• GSPN*OUFSO7JE.'-5<8BOH *$-3> CPVOEBSZBXBSFEBUBTFU • *O*OUFSO7JE.'-5 1Z4DFOF%FUFDU BVUPNBUJDBMMZTQMJUVOUSJNNFEWJEFPJOUPNVMUJQMF DMJQT 9
1SPQPTFE.FUIPE • 4UBHF*OTUSVDUJPO5VOJOH • "JNTUPBMJHOXJUIIVNBOJOUFOU • %JBMPHVFNBEFGSPNIJHIRVBMJUZUFNQPSBM SFMBUFEEBUBTFUTFOBCMF75JNF--.NPSF QSFDJTFFWFOUMPDBMJ[BUJPOBOEEFTDSJQUJPO •
%BUBTFU--.HFOFSBUFEWJEFPCBTFE EJBMPHVFTGSPN"DUJWJUZ/FU$BQUJPO <,SJTIOB *$$7> %7$ BOE%J%F.P <)FOESJDLT *$$7> 75( 10 Trained on Stage 2
1SPQPTFE.FUIPE • 1SPNQUTPGUSBJOJOH4UBHF 11 WJEFPGFBUVSF FNCFEEJOHMJTU XJMMCFJOTFSUFEIFSF TZTUFN QSPNQU
1SPQPTFE.FUIPE • 1SPNQUTPGUSBJOJOH4UBHF 12 WJEFPGFBUVSF FNCFEEJOHMJTU XJMMCFJOTFSUFEIFSF TZTUFN QSPNQU Q1
A1 Q2 A2 Q3 A3 Q4 A4 #PUI2BOE"BSF HFOFSBUFECZ--.
&YQFSJNFOUT • *NQMFNFOUBUJPO • --.7JDVOBW # # <;IFOH /FVS*14> •
-PTTGVODUJPOPSJHJOBMBVUPSFHSFTTJWFUSBJOJOHPCKFDUJWFTPGUIF--. DSPTT FOUSPQZ • *OTFDPOEBOEUIJSETUBHFT UIFZPOMZUSBJO-P3"QBSBNFUFST XJUIGSFF[JOH QSFUSBJOFEQBSBNFUFST • 5SBJOJOH359(16 • #BTFMJOFT • &YJTUJOH7JEFP--.T 7JEFP$IBU#<-J BS9JW> 7JEFP--B." #<;IBOH &./-1> 7JEFP$IBU(15 #<.BB[ "$-> 13
&YQFSJNFOUT • &WBMVBUFPO5XP Ћ UBTLT • 57( • %BUBTFU"DUJWJUZ/FU$BQUJPOTBOE$IBSBEFT45" •
.FUSJDNFBO*06 N*06 SFDBMM!N 3!N • %7$ • %BUBTFU"DUJWJUZ/FU$BQUJPOT • .FUSJD MPDBMJ[JOH U*P6 DBQUJPOJOH $*%&Sɾ.&5&03 DPNCJOBUJPO 40%"@D • 7JEFP%JBMPHVF • %BUBTFU7JEFP$IBU(15 CFODINBSL • .FUSJD(15BTTJHOTTDPSFT d UPBHFOFSBUFEUFYUXJUIBTQFDUT 14 7JEFP%JBMPHVF&WBMVBUJPO
&YQFSJNFOUT 75( %7$ 7JEFP%JBMPHVF 15 3FTVMUTTIPXT75JNF--.PVUQFSGPSNTCBTFMJOFTBDSPTT WBSJPVTWJEFPVOEFSTUBOEJOHUBTLT
&YQFSJNFOUT "CMBUJPO4UVEZ *NBHF5FYUQBJSJTNPSFTVJUBCMFSBUIFSUIBO7JEFP5FYUQBJSGPS4UBHF 7JTVBM"EBQUFSTIPVMECFGSFF[FBGUFS4UBHF -P3" QBSBNFUFSUSBJOFEPO4UBHFTIPVMECFBEEFEPO4UBHF 16
&YQFSJNFOUT 2VBMJUBUJWFSFTVMUTPOWJEFPEJBMPHVF • 75JNF--. JMMVTUSBUFTHPPEEJBMPHVFQFSGPSNBODF 17
$PODMVTJPO • 5IFZJOUSPEVDF75JNF--. B7JEFP--.DBQBCMFPGDPNQSFIFOEJOH NVMUJQMFFWFOUTXJUIJOBWJEFPBOEQSPWJEJOHQSFDJTFUFNQPSBM CPVOEBSJFT • 5IFZQSPQPTFBUISFFTUBHFUFNQPSBMBXBSFUSBJOJOHGSBNFXPSLXJUI UFNQPSBMSFMBUFE2VFTUJPO"OTXFSJOH MFWFSBHJOHNVMUJFWFOUEBUBTFUTGPS
5FNQPSBM7JEFP(SPVOEJOHBOE%FOTF7JEFP$BQUJPOJOH • &YUFOTJWFFYQFSJNFOUTEFNPOTUSBUFUIBU75JNF--.PVUQFSGPSNT FYJTUJOH7JEFP--.TTJHOJGJDBOUMZBDSPTTWBSJPVTWJEFPVOEFSTUBOEJOH UBTLT 18
5IBOLZPV 19
.ZDPODFSOT • $PNQBSJTPOXJUI%7$ 57(POMZNPEFMT • 1FSGPSNBODFPOPUIFSEPNBJO%7$EBUBTFUT • &WBMVBUJPOWBMJEJUZGPSQFSGPSNBODFPO7JEFP%JBMPHVF 20
zaemon1251hesty
badams
minecr
mayunak
nikkihalliwell
eitanlees
kneath
erikaheidi
garrettdimon
frankvandijk
akosma
sabderemane
deanohume
