Automatic façade recovery from single nighttime image

• Published in: Frontiers of Computer Science Vol. 14; no. 1; pp. 95 - 104
• Main Authors: ZHOU, Yi, GENG, Qichuan, ZHOU, Zhong, WU, Wei
• Format: Journal Article
• Language: English
• Published: Beijing Higher Education Press 01.02.2020; Springer Nature B.V
• Subjects: Computer Science; Facades; façade recovery; Human influences; Image analysis; Image reconstruction; lattice detection; Lattices; Light; Lighting; Methods; Night; nighttime images; Proposals; Recovery; Repetitive structures; Research Article; Urban environments; lattice detection; nighttime images; façade recovery
• ISSN: 2095-2228; 2095-2236
• DOI: 10.1007/s11704-017-6457-2

Nighttime images are difficult to process due to insufficient brightness, lots of noise, and lack of details. Therefore, they are always removed from time-lapsed image analysis. It is interesting that nighttime images have a unique and wonderful building features that have robust and salient lighting cues from human activities. Lighting variation depicts both the statistical and individual habitation, and it has an inherent man-made repetitive structure from architectural theory. Inspired by this, we propose an automatic nighttime façade recovery method that exploits the lattice structures of window lighting. First, a simple but efficient classification method is employed to determine the salient bright regions, which may be lit windows. Then we groupwindows into multiple lattice proposals with respect to façades by patch matching, followed by greedily removing overlapping lattices. Using the horizon constraint, we solve the ambiguous proposals problem and obtain the correct orientation. Finally, we complete the generated façades by filling in the missing windows. This method is well suited for use in urban environments, and the results can be used as a good single-view compensation method for daytime images. The method also acts as a semantic input to other learning-based 3D image reconstruction techniques. The experiment demonstrates that our method works well in nighttime image datasets, and we obtain a high lattice detection rate of 82.1% of 82 challenging images with a low mean orientation error of 12.1±4.5 degrees.