Deep filter banks for texture recognition and segmentation

• Published in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 3828 - 3836
• Main Authors: Cimpoi, Mircea, Maji, Subhransu, Vedaldi, Andrea
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 01.06.2015
• Subjects: Accuracy; Benchmark testing; Clutter; Construction; Feature extraction; Image recognition; Image segmentation; Segmentation; Shape; State of the art; Surface layer; Texture; Texture recognition
• ISSN: 1063-6919; 1063-6919; 2575-7075
• DOI: 10.1109/CVPR.2015.7299007

Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications. In this work we conduct a first study of material and describable texture attributes recognition in clutter, using a new dataset derived from the OpenSurface texture repository. Motivated by the challenge posed by this problem, we propose a new texture descriptor, FV-CNN, obtained by Fisher Vector pooling of a Convolutional Neural Network (CNN) filter bank. FV-CNN substantially improves the state-of-the-art in texture, material and scene recognition. Our approach achieves 79.8% accuracy on Flickr material dataset and 81% accuracy on MIT indoor scenes, providing absolute gains of more than 10% over existing approaches. FV-CNN easily transfers across domains without requiring feature adaptation as for methods that build on the fully-connected layers of CNNs. Furthermore, FV-CNN can seamlessly incorporate multi-scale information and describe regions of arbitrary shapes and sizes. Our approach is particularly suited at localizing "stuff" categories and obtains state-of-the-art results on MSRC segmentation dataset, as well as promising results on recognizing materials and surface attributes in clutter on the OpenSurfaces dataset.