Measuring the Significance of Policy Outputs with Positive Unlabeled Learning

Identifying important policy outputs has long been of interest to political scientists. In this work, we propose a novel approach to the classification of policies. Instead of obtaining and aggregating expert evaluations of significance for a finite set of policy outputs, we use experts to identify...

Full description

Saved in:
Bibliographic Details
Published inThe American political science review Vol. 115; no. 1; pp. 339 - 346
Main Authors ZUBEK, RADOSLAW, DASGUPTA, ABHISHEK, DOYLE, DAVID
Format Journal Article
LanguageEnglish
Published New York, USA Cambridge University Press 01.02.2021
Subjects
Automation
Citations
Classification
Construct Validity
Evaluators
Experts
Inferences
Law Enforcement
Law firms
Learning
Political science
Political scientists
Recall
Regulation
Subject specialists
Validation studies
Validity
Online AccessGet full text

Cover

More Information
Summary:Identifying important policy outputs has long been of interest to political scientists. In this work, we propose a novel approach to the classification of policies. Instead of obtaining and aggregating expert evaluations of significance for a finite set of policy outputs, we use experts to identify a small set of significant outputs and then employ positive unlabeled (PU) learning to search for other similar examples in a large unlabeled set. We further propose to automate the first step by harvesting “seed” sets of significant outputs from web data. We offer an application of the new approach by classifying over 9,000 government regulations in the United Kingdom. The obtained estimates are successfully validated against human experts, by forecasting web citations, and with a construct validity test.
Bibliography:SourceType-Scholarly Journals-1
ObjectType-Correspondence-2
content type line 14
ObjectType-Letter to the Editor-1
ISSN:0003-0554
1537-5943
DOI:10.1017/S000305542000091X