Diffusion-based data augmentation and hierarchical CLIP for real estate image annotation - Deng, Haojin; Zhang, Wandong; Yang, Yimin; Nejad, Eman (Pattern analysis and applications : PAA, 2025)
The critical role of real estate search engines in the economic domain underscores the necessity for a robust methodology in annotating the luxury level of room images. Contemporary challenges include inadequacies in room quality assessment, underutilization of deep network capacities, and insufficient annotated house images. This paper presents an innovative real estate image annotation model employing a multi-stage approach that integrates diffusion models with contrastive language-image pretraining (CLIP) networks. Initially, the diffusion network serves as a data augmentation technique, generating supplementary real estate images for training. Next, a hierarchical CLIP model categorizes images into different room types. Subsequently, multiple CLIP models assess the condition of each room, annotating them as either contemporary or standard. The final and most significant stage involves transferring knowledge from the larger CLIP model to a smaller, more efficient model using soft labeling. This approach enhances inferencing speed while maintaining high performance. Experimental results on a newly gathered real estate image dataset demonstrate the superior performance of the proposed approach compared to existing house image classification algorithms. Our demonstration code and checkpoints are available at https://huggingface.co/strollingorange/roomLuxuryAnnotater .