DETECLAP: Enhancing Audio-Visual Representation Learning with Object Information (Technical seminar on acoustic scene and event analysis)

2024/10/17に同志社大学にて開催されたTechnical seminar on acoustic scene and event analysisでのLINEヤフーの発表資料です。

Transcript

1. ˜
   -:
   $PSQPSBUJPO %&5&$-"1 &OIBODJOH
   "VEJP7JTVBM
   3FQSFTFOUBUJPO
   -FBSOJOH XJUI
   0CKFDU
   *OGPSNBUJPO 4IPUB
   /BLBEB 7JEFP
   BOBMZTJT
   UFBN
   .VMUJNPEBM
   "*
   %FQU
   -:
   $PSQPSBUJPO
2. ˜
   -:
   $PSQPSBUJPO "VEJPWJTVBM
   SFQSFTFOUBUJPO
   MFBSOJOH • -FBSO
   KPJOUMZ
   BVEJP
   BOE
   WJTVBM
   EBUB JO
   WJEFPT • "DRVJSF
   VTFGVM
   SFQSFTFOUBUJPOT
   GPS
   EPXOTUSFBN
   BVEJPWJTVBM
   UBTLT 8IBU
   JT
   BVEJPWJTVBM
   SFQSFTFOUBUJPO
   MFBSOJOH
   JOQVU
   WJEFP BVEJP
   
   WJTVBM "VEJP7JTVBM
   .PEFM PCUBJO SFQSFTFOUBUJPO GPS
   BVEJPWJTVBM
   UBTLT 
3. ˜
   -:
   $PSQPSBUJPO $POUSBTUJWF
   "VEJP7JTVBM
   .BTLFE
   "VUP&ODPEFS $"7."& • $POUSBTUJWF
   MFBSOJOH
   
   ."&
   GPS
   BVEJPWJTVBM
   QSFUSBJOJOH $"7."&
   <(POH
   *$-3`> BVEJP WJTVBM 
4. ˜
   -:
   $PSQPSBUJPO $POUSBTUJWF
   "VEJP7JTVBM
   .BTLFE
   "VUP&ODPEFS $"7."& • $POUSBTUJWF
   MFBSOJOH
   
   ."&
   GPS
   BVEJPWJTVBM
   QSFUSBJOJOH $"7."&
   <(POH
   *$-3`> BVEJP WJTVBM $POUSBTUJWF
   -PTT
   BVEJP
   BOE
   WJTVBM
   FNCFEEJOHT
   DMPTFS
   UPHFUIFS 3FDPOTUSVDUJPO
   -PTT EFWFMPQ
   VTFGVM
   SFQSFTFOUBUJPOT
   GPS
   EPXOTUSFBN
   UBTLT 
5. ˜
   -:
   $PSQPSBUJPO $"7."&
   <(POH
   *$-3`> BVEJP
   RVFSZ SFUSJFWFE
   WJTVBM BVEJP WJTVBM JOQVU QMBZJOH
   QJBOP DBUFHPSZ "VEJPWJTVBM
   SFUSJFWBM "VEJPWJTVBM
   DMBTTJGJDBUJPO %# SFUSJFWFE
   BVEJP WJTVBM
   RVFSZ %# QSFEJDU $POUSBTUJWF
   "VEJP7JTVBM
   .BTLFE
   "VUP&ODPEFS $"7."& • $POUSBTUJWF
   MFBSOJOH
   
   ."&
   GPS
   BVEJPWJTVBM
   QSFUSBJOJOH • "QQMJFE
   UP
   BVEJPWJTVBM
   SFUSJFWBM
   BOE
   DMBTTJGJDBUJPO 
6. ˜
   -:
   $PSQPSBUJPO *TTVF
   PG
   $"7."& RVFSZ JODPSSFDU
   SFTVMUT 'BJMVSF
   DBTFT
   PG
   $"7."& EPHT PCPF MJPO
   CJSE GMVUF
   IPSO  "
   MBDL
   PG
   BCJMJUZ
   UP
   SFDPHOJ[F
   GJOFHSBJOFE
   EFUBJMT • $"7."&
   DBQUVSFT
   SPVHI
   DBUFHPSZ
   lBOJNBMTz
   BOE
   lXJOE
   JOTUSVNFOUTz • #VU
   JU
   MBDLT
   GJOFHSBJOFE
   PCKFDU
   JOGPSNBUJPO
   lEPHTz
   BOE
   lPCPFz
7. ˜
   -:
   $PSQPSBUJPO )PX
   UP
   JNQSPWF
   $"7."&  /FFE
   GPS
   JOKFDU
   PCKFDU
   JOGPSNBUJPO
   UP
   $"7."& • -BDL
   GJOFHSBJOFE
   PCKFDU
   JOGPSNBUJPO ˠ JOKFDU
   PCKFDU
   JOGPSNBUJPO
   GSPN
   FYUFSOBM
   NPEFMT 1SPQPTFE
   NFUIPE
   %&5&$-"1 • *OUSPEVDF
   BO
   BVEJPWJTVBM
   MBCFM
   QSFEJDUJPO
   MPTT
   • &OIBODF
   PCKFDU
   SFDPHOJUJPO
   GPS
   QFSGPSNBODF
   JNQSPWFNFOU
8. ˜
   -:
   $PSQPSBUJPO  %&5"$-"1
   $"7."&
   XJUI
   "VEJP7JTVBM
   1SFEJDUJPO
   -PTT
9. ˜
   -:
   $PSQPSBUJPO $BMDVMBUF DPTJOF
   TJNJMBSJUJFT GPS
   MBCFM
   SFUSJFWBM 0CUBJO
   BVEJP
   FNCFEEJOH 0CUBJO
   UFYU
   FNCFEEJOH  "DRVJSJOH
   BVEJP
   MBCFMT
10. ˜
    -:
    $PSQPSBUJPO "DRVJSJOH
    WJTVBM
    MBCFMT %FUFDU
    PCKFDUT VTJOH
    PCKFDU
    EFUFDUPS 
11. ˜
    -:
    $PSQPSBUJPO .FSHJOH
    BVEJP
    BOE
    WJTVBM
    MBCFMT .FSHF BVEJP
    BOE
    WJTVBM
    MBCFMT 5XP
    UZQFT
    NFSHF
    PQFSBUJPO • "/% FMFNFOUXJTF
    MPHJDBM
    "/% • 03 FMFNFOUXJTF
    MPHJDBM
    03 
12. ˜
    -:
    $PSQPSBUJPO • 7((4PVOE • -BSHFTDBMF
    WJEFP
    EBUBTFU •  
    WJEFPT • 
    DBUFHPSZ • "VEJP4FU, • 4VCTFU
    PG
    "VEJP4FU •  
    WJEFPT • 
    DBUFHPSZ &YQFSJNFOU
    %BUBTFU 
13. ˜
    -:
    $PSQPSBUJPO • "VEJPWJTVBM
    SFUSJFWBM • 3FUSJFWF
    DPSSFTQPOEJOH
    BVEJPWJEFP
    GPS
    B
    HJWFO
    WJTVBMBVEJP
    JOQVU • "VEJPWJTVBM
    DMBTTJGJDBUJPO
    • 1SFEJDU
    DBUFHPSJFT
    VTJOH
    WJEFP
    BOE
    BVEJP
    BT
    JOQVUT &YQFSJNFOU
    5BTLT BVEJP
    RVFSZ SFUSJFWFE
    WJTVBM BVEJP WJTVBM JOQVU QMBZJOH
    QJBOP DBUFHPSZ "VEJPWJTVBM
    SFUSJFWBM "VEJPWJTVBM
    DMBTTJGJDBUJPO %# SFUSJFWFE
    BVEJP WJTVBM
    RVFSZ %# QSFEJDU 
14. ˜
    -:
    $PSQPSBUJPO &YQFSJNFOU
    3FUSJFWBM • %&5&$-"1
    PVUQFSGPSNT
    $"7."& • "VEJPCBTFE
    NPEFM
    JT CFUUFS
    UIBO
    7JTVBMCBTFE
    NPEFM • %&5&$-"1
    03
    TIPXT
    TVQFSJPS
    QFSGPSNBODF 
15. ˜
    -:
    $PSQPSBUJPO &YQFSJNFOU
    3FUSJFWBM • %&5&$-"1
    PVUQFSGPSNT
    $"7."& • "VEJPCBTFE
    NPEFM
    JT CFUUFS
    UIBO
    7JTVBMCBTFE
    NPEFM • %&5&$-"1
    03
    TIPXT
    TVQFSJPS
    QFSGPSNBODF
    
16. ˜
    -:
    $PSQPSBUJPO &YQFSJNFOU
    3FUSJFWBM • %&5&$-"1
    PVUQFSGPSNT
    $"7."& • "VEJPCBTFE
    NPEFM
    JT CFUUFS
    UIBO
    7JTVBMCBTFE
    NPEFM • %&5&$-"1
    03
    TIPXT
    TVQFSJPS
    QFSGPSNBODF 
17. ˜
    -:
    $PSQPSBUJPO &YQFSJNFOU
    $MBTTJGJDBUJPO • %&5&$-"1
    PVUQFSGPSNT
    $"7."&
    BOE
    "VEJPCBTFE
    NPEFMT • %&5&$-"1
    DBO
    FGGFDUJWFMZ
    BDRVJSF
    SFQSFTFOUBUJPOT
    GPS
    EPXOTUSFBN
    UBTLT 
18. ˜
    -:
    $PSQPSBUJPO &YQFSJNFOU
    2VBMJUBUJWF
    SFTVMU PCPF IPSTF PCPF IPSTF 
19. ˜
    -:
    $PSQPSBUJPO • 8F
    JOUSPEVDFE
    %&5&$-"1 • %&5&$-"1
    JNQSPWFT
    QFSGPSNBODF
    PO
    BVEJPWJTVBM
    SFUSJFWBM
    BOE
    DMBTTJGJDBUJPO
    UBTLT • %&5&$-"1
    BEWBODFT
    NVMUJNPEBM
    MFBSOJOH
    CZ
    FOIBODJOH
    UIF
    JOUFSQMBZ
    CFUXFFO
    BVEJP
    BOE
    WJTVBM
    NPEBMJUJFT
    UISPVHI
    PCKFDUBXBSF
    MFBSOJOH $PODMVTJPO