What does CNN learn ?

What does CNN learn ? Tomoki Tanimura B4, Jin Nakazawa
Lab, SFC, Keio University

3FBE1BQFST 6OEFSTUBOEJOH%FFQ*NBHF3FQSFTFOUBUJPOTCZ*OWFSUJOH5IFN "[email protected]"OESFB@7FEBMEJ *NBHF/FU5SBJOFE$//TBSFCJBTFEUPXBSETUFYUVSFJODSFBTJOHTIBQFCJBTJNQSPWFT BDDVSBDZBOESPCVTUOFTT 3PCFSU@(FJSIPT1BUSJDJB@3VCJTDI[email protected].BUUIJBT@#FUIHF 'FMJY@"@8JDINBOO8JFMBOE@#SFOEFM "QQSPYJNBUJOH$//TXJUI#BHPGMPDBM'FBUVSFTNPEFMTXPSLTTVSQSJTJOHMZXFMMPO
*NBHF/FU 8JFMBOE@#SFOEFM.BUUIJBT@#FUIHF &YQMPSJOHUIF0SJHJOTBOE1SFWBMFODFPG5FYUVSF#JBTJO$POWPMVUJPOBM/FVSBM/FUXPSLT ,BUIFSJOF@-@)FSNBOO4JNPO@,PSOCMJUI

"CTUSBDU 8FUIJOLUIBU$//MFBSOUIFHMPCBMTUSVDUVSFPGUIFJNBHF FH(MPCBMTIBQFPGUIFPCKFDU 4USVDUVSFPGUIFFEHF #VU (FJSIPTFUBMSFWFBMTUIBU$//USBJOFEPO*NBHF/FUIBWFUIFUFYUVSFCJBT $MBTTJGZUIFJNBHFCBTFEPOUIFUFYUVSFPGUIFJNBHF 4P UIFQFSGPSNBODFPG#BH/FUXIJDIJTSFTUSJDUFEUPTFFPOMZUIFMPDBM GFBUVSFTJTDPNQFUJUJWFXJUI%FFQFS/FUXPSLMJLF3FT/FU
&NQJSJDBMMZDPOpSNUIBU$//MFBSOUIFMPDBMGFBUVSFTMJLFUFYUVSF )FSNBOOFUBMFNQJSJDBMMZTIPXUIBU$//JUTFMGEPFTO`UIBWFUIFJOEVDUJWF CJBTGPSUFYUVSFBOEpOEUIFPSJHJOPGUFYUVSFCJBTJO$// 5IFPCTFSWFEUFYUVSFCJBTJTBUUSJCVUFEUPQSFQSPDFTTFT UBTLT BOEMFBSOJOHSBUFT

Preliminaries

8IBUJT$// $POWPMVUJPOBM/FVSBM/FUXPSL $// 5IJTJTEFWFMPQFEGPSJNBHFQSPDFTTJOH Let’t try NN ! https://scs.ryerson.ca/~aharley/vis/conv/

3FQSFTFOUBUJWF.PEFMTCBTFEPO$// 7(( 3FT/FU /"4/FU &⒏DJFOU/FU "MFY/FU

5IFGPSXBSEQBTTPG$// 5ISFFDMBTTDMBTTJpDBUJPOUBTL <bNPOLFZ` bQFSTPO` bDIBJS`> *OQVU*NBHF < > 5IJTJNBHFJTBNPOLFZ

*NBHF/FU$MBTTJpDBUJPO DMBTTDMBTTJpDBUJPO4P5" UPQ

What does learn CNN? [Understanding Deep Image Representations by Inverting
Them]

$//MFBSOQIPUPTMJLFGFBUVSFT *OQVU*NBHFBOEGFBUVSFTFODPEFECZ$// 'FBUVSFTPGFBDIMBZFS %FFQFS NPSFBCTUSBDUPGHFOFSBMDPOUFOUPGUIFDMBTT TIBMMPX EFFQ

)PXEPXFDIFDLUIFMFBSOFEGFBUVSFT 0SJHJOBM*NBHF 'FBUVSF0

)PXEPXFDIFDLUIFMFBSOFEGFBUVSFT 0SJHJOBM*NBHF /PJTF*NBHF 'FBUVSF/ 'FBUVSF0

)PXEPXFDIFDLUIFMFBSOFEGFBUVSFT $BMDVMBUFBOENJOJNJ[FUIFMPTT 0SJHJOBM*NBHF /PJTF*NBHF 'FBUVSF/ 'FBUVSF0

)PXEPXFDIFDLUIFMFBSOFEGFBUVSFT $BMDVMBUFBOENJOJNJ[FUIFMPTT 6QEBUFUIFOPJTFJNBHF GPSNBLJOHUIFGFBUVSF/TJNJMBSUPGFBUVSF0 0SJHJOBM*NBHF /PJTF*NBHF 'FBUVSF/ 'FBUVSF0

)PXEPXFDIFDLUIFMFBSOFEGFBUVSFT $BMDVMBUFBOENJOJNJ[FUIFMPTT 6QEBUFUIFOPJTFJNBHFJUFSBUJWFMZ 0SJHJOBM*NBHF /PJTF*NBHF 'FBUVSF/ 'FBUVSF0

)PXEPXFDIFDLUIFMFBSOFEGFBUVSFT -PTTJTFOPVHITNBMM 6QEBUFEUIFOPJTFJNBHFTPNBOZUJNFT 0SJHJOBM*NBHF *NBHFSFQSFTFOUJOH UIFGFBUVSF/ 'FBUVSF/ 'FBUVSF0

'PSNVMBUJPOPG*OWFSUJOH3FQSFTFOUBUJPOT 5IJTJTUIFGPSNVMBUJPOPGl*OWFSUJOH3FQSFTFOUBUJPOTz YJTUIFPSJHJOBMJOQVUJNBHF Y TUBSUGSPNOPJTFJNBHF UIFODIBOHFUPUIFJNBHFSFQSFTFOUJOHUIFGFBUVSF CZPQUJNJ[BUJPO JTMPTTGVODUJPOTVDIBT-OPSN JTUIF// JTUIFSFHVMBSJ[BUJPOUFSN
l( . , . ) Φ( . ) ℛ( . )

%FFQFSGFBUVSFTIBTNPSFJOGPSNBUJPO 5IFSFDPOTUSVDUJPOPGYDFOUFSQBUDIPGFBDIGFBUVSFNBQT %FFQGFBUVSFTIBTSJDIJOGPSNBUJPOBUUIFDFOUFSPGGFBUVSFNBQ

"MFY/FUIBTUXPUZQFTPGGFBUVSFTJOJUTFMG "MFY/FUIBTUXPQJQFMJOFJOBOFUXPSL &BDIQJQFMJOFMFBSOEJ⒎FSFOUGFBUVSFTFWFOUIPVHIUIFSFJTOP SFHVMBSJ[BUJPO 0OFGPSMPXGSFRVFODZ FHDPMPS "OPUIFSGPSIJHIPOF FHJOUFOTJUZ 0OFQJQFMJOF "OPUIFSQJQFMJOF

4VNNBSZPGUIJTQBQFS $//TMFBSOUIFQIPUPMJLFGFBUVSFT *U`TTJNJMBSUPIVNBOT -FBSOFEGFBUVSFTBSFEJ⒎FSFOUJOMBZFST 4IBMMPXMBZFSMFBSOMPDBMGFBUVSFT FHDPMPS UFYUVSF FUD %FFQ-BZFSMFBSONPSFHMPCBMGFBUVSFT
FHHFOFSBMSFQSFTFOUBUJPOPG UIFDMBTTJNBHF

Are CNNs like human eye? [ImageNet-Trained CNNs are biased towards
texture; increasing shape bias improves accuracy and robustness]

$//IBTUIFUFYUVSFCJBT *NBHF/FUUSBJOFE$//DMBTTJGZJNBHFTCBTFEPOUIFUFYUVSFSBUIFS UIBOUIFTIBQF

$//TBSF/05IVNBOFZF 5IFVTFGVMJOGPSNBUJPOGPS$//JTEJ⒎FSFOUGSPN)VNBO $//MPPLTUIFUFYUVSFSBUIFSUIBOUIFTIBQF )VNBOMPPLTUIFTIBQFTSBUIFSUIBOUIFUFYUVSF

4IBQFPS5FYUVSF5FTU 4IBQFBOE5FYUVSFBSFCFMPOHFEUPEJ⒎FSFOUDMBTTFTFBDIPUIFS )VNBOTBOE$//TBTTJHOBOJNBHFUPBTJOHMFDMBTT )VNBOT $//T

$BOXFUFBDIBTIBQFUP$// $SFBUFUIF4UZMJ[FE*NBHF/FU 4IBQFJTSFUBJOFE CVU5FYUVSFJTDPSSVQUFE 6TF"EB*/4UZMF5SBOTGFSGPSHFOFSBUJOHUIJTEBUBTFU

5IFSFTVMUTUPUSBJO$//PO4UZMJ[FE*NBHF/FU 8FDBOUFBDI$//TBTIBQF )VNBOTIFBWJMZSFMZPOUIFTIBQF 4*/USBJOFE$//TSFMZPOUIFTIBQF  SBUIFSUIBOUIFUFYUVSF */USBJOFE$//TSFMZPOUIFUFYUVSF

4VNNBSZPGUIJTQBQFS )VNBOTMJLFTIBQF CVU$//TMJLFUFYUVSF )VNBOTSFMZPOTIBQFXIFODMBTTJGZJOHJNBHFT $//TSFMZPOUFYUVSF 8FDBOUFBDITIBQFUP$//TVTJOHXFMMEFTJHOFEEBUBTFU 1SPQPTFUIF4UZMJ[FE*NBHF/FU 4IBQFJTPSJHJOBMDMBTT 5FYUVSFJTOPUSFMBUFEUPUIFPSJHJOBMDMBTT DPSSVQUFE

CNN : “No! It’s not my fault!” [Exploring the Origins
and Prevalence of Texture Bias in Convolutional Neural Networks]

5FYUVSFCJBTJTOPUUIFJOEVDUJWFCJBTPG$//T $//TDBOFBTJMZMFBSOUFYUVSF 5IFPSJHJOPGUFYUVSFCJBT %BUBTFUBOE0CKFDUJWFGVODUJPO "SDIJUFDUVSFPGUIFNPEFM %BUB"VHNFOUBUJPO )ZQFSQBSBNFUFST

)PXUPpOEUIFPSJHJO 6TFUIFTFEBUBTFU (FJSIPT`TPOFJTNBJOMZVTFEJOUIFFYQFSJNFOU

*TJUEJ⒏DVMUGPS$//TUPMFBSOTIBQF $MBTTJpDBUJPO5BTLGPSUIBUUISFFEBUBTFU $//TDBOMFBSOBTIBQFBUMFBTUBTSFBEJMZBTBUFYUVSF

5PXIBUFYUFOUBSFTIBQFBOEUFYUVSFSFQSFTFOBUFEJOUIF$// -JOFBS$MBTTTJpDBUJPOGPSTIBQFBOEUFYUVSFPO(FJSIPTEBUBTFU *OQVUGFBUVSFTFODPEFECZ*NBHF/FUUSBJOFE$// *NBHF/FUUSBJOFE$//TIBWFCPUIPGTIBQFBOEUFYUVSFSFQSFTFOUBUJPO

%PFTPCKFDUJWFB⒎FDUTIBQFCJBT -JOFBS$MBTTJpDBUJPOGPSTIBQFBOEUFYUVSFPO(FJSIPTEBUBTFU *OQVUGFBUVSFTFODPEFECZ$//TUSBJOFEPOWBSJPVTPCKFDUJWFT 0CKFDUJWFT3PUBUJPO &YBNQMFS #JH#J("/ 3FTVMUT 3PUBUJPONPSFB⒎FDUTIBQFCJBT

%PFTBSDIJUFDUVSFB⒎FDUTIBQFCJBT .FBTVSFUIFTIBQFCJBTVTJOH(FJSIPTEBUBTFUGPS*NBHF/FUUSBJOFE $//T 'JOEUIF$PSSFMBUJPOCFUXFFOTIBQFCJBTBOEUIF*NBHF/FUBDDVSBDZ

%PFTUIFUSBJOJOHQSPDFTTDPOUSJCVUFUPUFYUVSFCJBT 3BOEPNDSPQQSFQSPDFTTJOHCJBTNPEFMTUPXBSETUFYUVSF :FT0GDPBSTF 8IFOBQQMZJOHUIFDFOUFSDSPQJOUIFUSBJOJOH $//TIBWFNPSFTIBQF CJBTSBUIFSUIBOUFYUVSF

)PXBCPVU)ZQFSQBSBNFUFST *OWFTUJHBUFUIFSFMBUJPOUPMFBSOJOHSBUFBOEXFJHIUEFDBZXJUI TIBQFPSUFYUVSFCJBT -BSHFSMS $//TIBTNPSFTIBQFCJBTBOEWJDFWFSTB 8EIBTBMTPTJNJMBSQSPQFSUZBTMS

4VNNBSZPGUIJTQBQFS 5FYUVSFCJBTJTOPUUIFJOEVDUJWFCJBTPG$//T 5IFTFGBDUPSTJOqVFODFUFYUVSFCJBTBOE8FDBOBEKVTUUIFN 0CKFDUJWF'VODUJPO "SDIJUFDUVSFPGUIFNPEFM %BUB"VHNFOUBUJPO )ZQFSQBSBNFUFST 5IFSFJTUIFDPOVOESVN 4IBQFCJBTDPSSFMBUFXJUI*NBHF/FUBDDVSBDZ *NBHF/FUTUBOEBSEUSBJOFE$//TIBWFUFYUVSFCJBT

As a result, What does CNN learn?

*TJUEFQFOETPOUIFUSBJOJOHFOWJSPONFOU 6OEFSTUBOEJOH%FFQ*NBHF3FQSFTFOUBUJPOTCZ*OWFSUJOH5IFN $//TMFBSOQIPUPMJLFGFBUVSFT -FBSOFEGFBUVSFTBSFEJ⒎FSFOUJOMBZFST 4IBMMPX-PDBM %FFQ(MPCBM *NBHF/FUUSBJOFE$//TBSFCJBTFEUPXBSETUFYUVSFJODSFBTJOHTIBQFCJBT
JNQSPWFTBDDVSBDZBOESPCVTUOFTT )VNBOTMJLFTIBQF CVU$//TMJLFUFYUVSF &YQMPSJOHUIF0SJHJOTBOE1SFWBMFODFPG5FYUVSF#JBTJO$POWPMVUJPOBM/FVSBM /FUXPSLT 5FYUVSFCJBTJTOPUUIFJOEVDUJWFCJBTPG$//T5SBJOJOHFOWJSPONFOUTB⒎FDUUIJTCJBT #VU UIFSFJTUIFDPOVOESVNBOEPOMZFNQJSJDBMSFTVMUTCBTFEPOUIF(BJSIPTEBUBTFU

What does CNN learn ?

What does CNN learn ?

More Decks by tanimutomo

Other Decks in Research

Featured

Transcript