Sketch123: Multi-spectral channel cross attention for sketch-based 3D generation via diffusion models

• Published in: Computer aided design Vol. 185; p. 103896
• Main Authors: Xu, Zhentong, Zeng, Long, Zhao, Junli, Wang, Baodong, Pan, Zhenkuan, Liu, Yong-Jin
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.08.2025
• Subjects: Attention mechanism; Diffusion model; Sketch-based generation; Spectral channel attention; Attention mechanism; Sketch-based generation; Spectral channel attention; Diffusion model
• ISSN: 0010-4485
• DOI: 10.1016/j.cad.2025.103896

With the development of generative techniques, sketch-driven 3D reconstruction has gained substantial attention as an efficient 3D modeling technique. However, challenges remain in extracting detailed features from sketches, representing local geometric structures, and ensuring generated fidelity and stability. To address these issues, in this paper we propose a multi-spectral channel cross-attention model for sketch reconstruction, which leverages the complementary strengths of frequency and spatial domains to capture multi-level sketch features. Our method employs a two-stage diffusion generation mechanism, additionally, a Sparse Feature Enhancement Module (SFE) replaces traditional down-sampling, reducing feature loss and enhancing detail preservation and noise suppression through a Laplace voxel smoothing operator. The Wasserstein distance introduced and integrated as part of the loss function, stabilizes the generative process using optimal transport theory to support high-quality 3D model reconstruction. Extensive experiments verify that our model surpasses state-of-the-art methods in terms of generation accuracy, local control, and generalization ability, providing an efficient, precise solution for transforming sketches into 3D models. •Optimal transport-guided diffusion model ensures consistent, detailed 3D generation.•Multi-spectral channel cross-attention integrates spatial and frequency features.•Sparse feature enhancement and Laplace voxel smoothing boost reconstruction quality.