Magic Layouts: Structural Prior for Component Detection in User Interface Designs
We present Magic Layouts; a method for parsing screen-shots or hand-drawn sketches of user interface (UI) layouts. Our core contribution is to extend existing detectors to exploit a learned structural prior for UI designs, enabling robust detection of UI components; buttons, text boxes and similar....
Saved in:
Published in | 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 15804 - 15813 |
---|---|
Main Authors | , , |
Format | Conference Proceeding |
Language | English |
Published |
IEEE
01.06.2021
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | We present Magic Layouts; a method for parsing screen-shots or hand-drawn sketches of user interface (UI) layouts. Our core contribution is to extend existing detectors to exploit a learned structural prior for UI designs, enabling robust detection of UI components; buttons, text boxes and similar. Specifically we learn a prior over mobile UI layouts, encoding common spatial co-occurrence relationships between different UI components. Conditioning region proposals using this prior leads to performance gains on UI layout parsing for both hand-drawn UIs and app screen-shots, which we demonstrate within the context an interactive application for rapidly acquiring digital prototypes of user experience (UX) designs. |
---|---|
ISSN: | 2575-7075 |
DOI: | 10.1109/CVPR46437.2021.01555 |