River state classification combining patch-based processing and CNN

• Published in: PloS one Vol. 15; no. 12; p. e0243073
• Main Authors: Oga, Takahiro, Harakawa, Ryosuke, Minewaki, Sayaka, Umeki, Yo, Matsuda, Yoko, Iwahashi, Masahiro
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 03.12.2020; Public Library of Science (PLoS)
• Subjects: Artificial neural networks; Biology and Life Sciences; Cameras; Classification; Computer and Information Sciences; Computer science; Databases, Factual; Datasets; Earth Sciences; Ecology and Environmental Sciences; Engineering; Engineering and Technology; Environmental Monitoring - methods; Environmental risk; Flood damage; Flood risk; Flood warnings; Floods; Heavy rainfall; Image classification; Image processing; Image Processing, Computer-Assisted - methods; Image segmentation; Information processing; Japan; Labels; Methods; Natural history; Neural networks; Neural Networks, Computer; Rainfall; Research and Analysis Methods; Rivers; Semantics; Sensors; Surveillance; Training; Japan
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0243073

This paper proposes a method for classifying the river state (a flood risk exists or not) from river surveillance camera images by combining patch-based processing and a convolutional neural network (CNN). Although CNN needs much training data, the number of river surveillance camera images is limited because flood does not frequently occur. Also, river surveillance camera images include objects that are irrelevant to the flood risk. Therefore, the direct use of CNN may not work well for the river state classification. To overcome this limitation, this paper develops patch-based processing for adjusting CNN to the river state classification. By increasing training data via the patch segmentation of an image and selecting patches that are relevant to the river state, the adjustment of general CNNs to the river state classification becomes feasible. The proposed patch-based processing and CNN are developed independently. This yields the practical merits that any CNN can be used according to each user's purposes, and the maintenance and improvement of each component of the whole system can be easily performed. In the experiment, river state classification is defined as the following problems using two datasets, to verify the effectiveness of the proposed method. First, river images from the public dataset called Places are classified to images with Muddy labels and images with Clear labels. Second, images from the river surveillance camera in Nagaoka City, Japan are classified to images captured when the government announced heavy rain or flood warning and the other images.