Sequential Feature Filtering Classifier

The Ensemble and mixture of expertise method is the most intuitive and simple way to improve performance in the field of recognition using convolutional neural networks (CNNs). However, Ensemble is difficult to apply in real-time operation applications because the amount of computational overhead and parameters increase in proportion to the number of models. In another, a mixture of expertise that extracts various expertise and combines it is cumbersome to apply because it requires a large change in the network. In this study, we propose Sequential Feature Filtering Classifier (FFC), a simple but effective classifier for CNNs. Using sequential LayerNorm and ReLU, FFC zeroes out low-activation units and preserves high-activation units. The sequential feature filtering process generates multiple features, which are transmitted to a shared classifier, yielding multiple outputs (multiple expertise). FFC can be applied to any CNN with a classifier and it significantly improves the performance with negligible overhead. In this study, the efficacy of FFC is validated extensively on various tasks—ImageNet-1K classification, MS COCO detection, Cityscapes segmentation, and HMDB51 action recognition. Moreover, it is empirically established that FFC can further improve performances using additional techniques, including attention modules. Code is available at https://github.com/seominseok0429/Sequential-Feature-Filtering-Classifier.