Equipping sketch patches with context-aware positional encoding for graphic sketch representation

• Published in: Computer vision and image understanding Vol. 258; p. 104385
• Main Authors: Zang, Sicong, Fang, Zhijun
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.07.2025
• Subjects: Contextual relationships; Graph convolutional network; Graphic sketch representation; Positional encoding; Sketch drawing order; Contextual relationships; Graphic sketch representation; Graph convolutional network; Sketch drawing order; Positional encoding
• ISSN: 1077-3142
• DOI: 10.1016/j.cviu.2025.104385

When benefiting graphic sketch representation with sketch drawing orders, recent studies have linked sketch patches as graph edges by drawing orders in accordance to a temporal-based nearest neighboring strategy. However, such constructed graph edges may be unreliable, since the contextual relationships between patches may be inconsistent with the sequential positions in drawing orders, due to variants of sketch drawings. In this paper, we propose a variant-drawing-protected method by equipping sketch patches with context-aware positional encoding (PE) to make better use of drawing orders for sketch learning. We introduce a sinusoidal absolute PE to embed the sequential positions in drawing orders, and a learnable relative PE to encode the unseen contextual relationships between patches. Both types of PEs never attend the construction of graph edges, but are injected into graph nodes to cooperate with the visual patterns captured from patches. After linking nodes by semantic proximity, during message aggregation via graph convolutional networks, each node receives both semantic features from patches and contextual information from PEs from its neighbors, which equips local patch patterns with global contextual information, further obtaining drawing-order-enhanced sketch representations. Experimental results indicate that our method significantly improves sketch healing and controllable sketch synthesis. •Rethink the attendance of drawing orders in graphic sketch representation.•Embed contextual information among sketch strokes by positional encoding.•Inject drawing orders into graph nodes in graphic sketch representation learning.•Significantly improve sketch healing and controllable sketch synthesis.