A physical model-based approach to detecting sky in photographic images

• Published in: IEEE transactions on image processing Vol. 11; no. 3; pp. 201 - 212
• Main Authors: Jiebo Luo, Etz, S.P.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.03.2002; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Artificial intelligence; Backpropagation algorithms; Cathode ray tubes; Classification; Color; Computer science; control theory; systems; Content based retrieval; Exact sciences and technology; Extraction; Image processing; Image recognition; Image retrieval; Information, signal and communications theory; Laboratories; Layout; Multi-layer neural network; Multilayers; Neural networks; Pattern recognition. Digital image processing. Computational geometry; Physics computing; Signal processing; Signatures; Sky; Studies; Telecommunications and information theory; Sky; Backpropagation; Photographic image; Image processing; Neural network; Algorithm; Image segmentation; Physical model; Object detection; Feature extraction; Gradient method; Desaturation; Image classification
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/83.988954

Sky is among the most important subject matter frequently seen in photographic images. We propose a model-based approach consisting of color classification, region extraction, and physics-motivated sky signature validation. First, the color classification is performed by a multilayer backpropagation neural network trained in a bootstrapping fashion to generate a belief map of sky color. Next, the region extraction algorithm automatically determines an appropriate threshold for the sky color belief map and extracts connected components. Finally, the sky signature validation algorithm determines the orientation of a candidate sky region, classifies one-dimensional (1-D) traces within the region based on a physics-motivated model, and computes the sky belief of the region by the percentage of traces that fit the physics-based sky trace model. A small-scale, yet rigorous test has been conducted to evaluate the algorithm performance. With approximately half of the images containing blue sky regions, the detection rate is 96% with a false positive rate of 2% on a per image basis.