Curating Training Data for Reliable Large-Scale Visual Data Analysis: Lessons from Identifying Trash in Street View Imagery

Visual data have dramatically increased in quantity in the digital age, presenting new opportunities for social science research. However, the extensive time and labor costs to process and analyze these data with existing approaches limit their use. Computer vision methods hold promise but often req...

Full description

Saved in:
Bibliographic Details
Published inSociological methods & research Vol. 52; no. 3; pp. 1155 - 1200
Main Authors Hwang, Jackelyn, Dahir, Nima, Sarukkai, Mayuka, Wright, Gabby
Format Journal Article
LanguageEnglish
Published Los Angeles, CA SAGE Publications 01.08.2023
SAGE PUBLICATIONS, INC
Subjects
Artificial Intelligence
Data
Data Analysis
Data Interpretation
Imagery
Labor costs
Municipalities
Reliability
Sanitation
Social research
Social Science Research
Sociology
Training
Visual Aids
Massachusetts (Boston)
Michigan (Detroit)
California (Los Angeles)
visual data
systematic social observation
crowdsourcing
computer vision
urban sociology
Online AccessGet full text

Cover

More Information
Summary:Visual data have dramatically increased in quantity in the digital age, presenting new opportunities for social science research. However, the extensive time and labor costs to process and analyze these data with existing approaches limit their use. Computer vision methods hold promise but often require large and nonexistent training data to identify sociologically relevant variables. We present a cost-efficient method for curating training data that utilizes simple tasks and pairwise comparisons to interpret and analyze visual data at scale using computer vision. We apply our approach to the detection of trash levels across space and over time in millions of street-level images in three physically distinct US cities. By comparing to ratings produced in a controlled setting and utilizing computational methods, we demonstrate generally high reliability in the method and identify sources that limit it. Altogether, this approach expands how visual data can be used at a large scale in sociology.
ISSN:0049-1241
1552-8294
DOI:10.1177/00491241231171945