• S-cells (convolution) feature extraction • C-cells (avg. pooling) robustness to positional deviation • Self-organizing like training • NOT back propagation 4/17
non-linear combinations of feature maps • Global Average Pooling one feature map for each class no Fully Connected layers • Small model size 29 [MB] for ImageNet MLP Softmax 7/17
• convolution • max pooling • activation (ReLU) • Fully Connected • softmax • Sequence of small convolutions • 3*3 spatial convolutions • Relatively large parameters • large channels at the early stages • many Fully Connected layers 8/17
concatenation capture different features efficiently • mainly 3*3 convolution coming from the VGG architecture • 1*1 convolution reduce the number of channels • Global Average Pooling and Fully Connected • balance accuracy and model size • Good performance! 9/17
Fire module squeeze channels to reduce computational costs • Deep compression lighten model size sparse weight, weight quantization, Huffman coding • Small model for 6 bit data, the model size is 0.47 [MB] ! 1*1 squeeze 1*1 expand 3*3 expand
the early stages • asymmetric encoder-decoder structure • PReLU • small model ~ 1[MB] • Encoder can be used as CNN • Global Max Pooling encoder decoder input 3 × 512 × 512 12/17
convolution • batch normalization • Advantages of Complex value • biological & signal processing aspects can express firing rate & relative timing detailed description of objects • parameter efficient 2^(depth) efficient than real value