Detecting user interface elements in robotic process automation using convolutional neural networks

Graphical elements in a user interface (UI) may be detected in robotic process automation (RPA) using convolutional neural networks (CNNs). Such processes may be particularly well-suited for detecting graphical elements that are too small to be detected using conventional techniques. The accuracy of...

Full description

Saved in:
Bibliographic Details
Main Authors Bobolea, Dragos, Neagovici, Mircea, Tudor, Virgil, Adam, Stefan
Format Patent
LanguageEnglish
Published 07.03.2023
Subjects
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
PHYSICS
Online AccessGet full text

Cover

More Information
Summary:Graphical elements in a user interface (UI) may be detected in robotic process automation (RPA) using convolutional neural networks (CNNs). Such processes may be particularly well-suited for detecting graphical elements that are too small to be detected using conventional techniques. The accuracy of detecting graphical elements (e.g., control objects) may be enhanced by providing neural network-based processing that is robust to changes in various UI factors, such as different resolutions, different operating system (OS) scaling factors, different dots-per-inch (DPI) settings, and changes due to UI customization of applications and websites, for example.
Bibliography:Application Number: US202117209601