Machine-assisted annotation of forensic imagery

• Published in: arXiv.org
• Main Authors: Mousavi, Sara, Nabati, Ramin, Kleeschulte, Megan, Mockus, Audris
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 28.02.2019
• Subjects: Annotations; Domains; Image segmentation; Machine learning
• ISSN: 2331-8422

Image collections, if critical aspects of image content are exposed, can spur research and practical applications in many domains. Supervised machine learning may be the only feasible way to annotate very large collections, but leading approaches rely on large samples of completely and accurately annotated images. In the case of a large forensic collection, we are aiming to annotate, neither the complete annotation nor the large training samples can be feasibly produced. We, therefore, investigate ways to assist manual annotation efforts done by forensic experts. We present a method that can propose both images and areas within an image likely to contain desired classes. Evaluation of the method with human annotators showed highly accurate classification that was strongly helped by transfer learning. The segmentation precision (mAP) was improved by adding a separate class capturing background, but that did not affect the recall (mAR). Further work is needed to both increase the accuracy of segmentation and enhances prediction with additional covariates affecting decomposition. We hope this effort to be of help in other domains that require weak segmentation and have limited availability of qualified annotators.