336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 Visualizing and Understanding Convolutional Neural Networks Val Val Test Error % Top-1 Top-5 Top-5 (Krizhevsky et al., 2012), 1 convnet 40.7 18.2 (Krizhevsky et al., 2012), 5 convnets 38.1 16.4 16.4 (Krizhevsky et al., 2012), 1 convnets* 39.0 16.6 (Krizhevsky et al., 2012), 7 convnets* 36.7 15.4 15.3 Our replication of (Krizhevsky et al., 2012), 1 convnet 40.5 18.1 1 convnet as per Fig. 2 38.3 16.4 16.5 5 convnets as per Fig. 2 36.6 15.3 15.3 Table 2. ImageNet 2012 classification error rates. The ⇤ indicates models that were trained on both ImageNet 2011 and 2012 training sets with an additional convolution layer. When we combine multiple models, we obtain a test error of 15 . 3%, which matches the absolute best per- formance on this dataset, despite only using the much smaller 2012 training set. We note that this error is almost half that of the top non-convnet entry in the ImageNet 2012 classification challenge, which obtained 26 . 1% error. 3.1. Training Details The models were trained on the ImageNet 2012 train- ing set (1.3 million images, spread over 1000 di↵erent classes). Each RGB image was preprocessed by resiz- ing the smallest dimension to 256, cropping the center 256x256 region, subtracting the per-pixel mean (across all images) and then using 10 di↵erent sub-crops of size 224x224 (corners + center with(out) horizontal flips). change in the image from which the strongest act vation originates. Due to space constraints, only randomly selected subset of feature maps are visua ized and zooming is needed to see the details clearly As expected, the first layer filters consist of Gabor and low-frequency color. The 2nd layer features ar more complex, corresponding to conjunctions of edge and color patterns. The 3rd layer features show large image parts. Within a given feature projection, signif icant variations in contrast can be seen, showing whic parts of the image contribute most to the activatio and thus are most discriminative, e.g. the lips and eye on the persons face (Row 12). The visualization from the 4th and 5th layer show activations that respon to complex objects. Note that little of the scene back ground is reconstructed, since it is irrelevant to pre dicting the class. 4.2. Feature Invariance Fig. 4 shows feature visualizations from our mode once training is complete. However, instead of show ing the single strongest activation for a given fea ture map, we show the top 9 activations. Projectin each separately down to pixel space reveals the di↵er ent structures that excite a given feature map, henc showing its invariance to input deformations. Laye Visualizing and Understanding Convo Train Val Val Error % Top-1 Top-1 Top-5 Our replication of (Krizhevsky et al., 2012), 1 convnet 35.1 40.5 18.1 Removed layers 3,4 41.8 45.4 22.1 Removed layer 7 27.4 40.0 18.4 Removed layers 6,7 27.4 44.8 22.4 Removed layer 3,4,6,7 71.1 71.3 50.1 Adjust layers 6,7: 2048 units 40.3 41.7 18.8 Adjust layers 6,7: 8192 units 26.8 40.0 18.1 Our Model (as per Fig. 3) 33.1 38.4 16.6 Adjust layers 6,7: 2048 units 38.2 40.2 17.6 Adjust layers 6,7: 8192 units 22.0 38.8 17.0 Adjust layers 3,4,5: 512,1024,512 maps 18.8 37.5 16.0 Adjust layers 6,7: 8192 units and Layers 3,4,5: 512,1024,512 maps 10.0 38.3 16.9 Table 3. ImageNet 2012 classification error rates with var- ious architectural changes to the model of (Krizhevsky et al., 2012) and our ImageNet model. Figure 9 number 6 trainin
hakka_labs
ivargrimstad
hollycummins
manfredsteyer
kakao
tsoshiro
10tera
maaretp
usagimaru
qnighy
tutinoko
dero1to
keithpitt
chriscoyier
keathley
shlominoach
bkeepers
scottboms
speakerdeck
phodgson
tammyeverts
jakevdp
bermonpainter
![Krizhevsky et al. [NIPS2012] • 7 hidden layers, 650,000 neurons,](https://files.speakerdeck.com/presentations/d9ba942209d9428880a028549da40d21/slide_4.jpg){kind=link}
( • Feed[forward(processing(only(in(monkey/humans( 6 V4 Cortex Extrapolation](https://files.speakerdeck.com/presentations/d9ba942209d9428880a028549da40d21/slide_9.jpg){kind=link}
![Caltech 256 [3] L. Bo, X. Ren, and D. Fox.](https://files.speakerdeck.com/presentations/d9ba942209d9428880a028549da40d21/slide_62.jpg){kind=link}
