Using deep learning to predict outcomes of legal appeals better than human experts: A study with data from Brazilian federal courts
Legal scholars have been trying to predict the outcomes of trials for a long time. In recent years, researchers have been harnessing advancements in machine learning to predict the behavior of natural and social processes. At the same time, the Brazilian judiciary faces a challenging number of new c...
Saved in:
Published in | PloS one Vol. 17; no. 7; p. e0272287 |
---|---|
Main Authors | , |
Format | Journal Article |
Language | English |
Published |
San Francisco
Public Library of Science
28.07.2022
Public Library of Science (PLoS) |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Legal scholars have been trying to predict the outcomes of trials for a long time. In recent years, researchers have been harnessing advancements in machine learning to predict the behavior of natural and social processes. At the same time, the Brazilian judiciary faces a challenging number of new cases every year, which generates the need to improve the throughput of the justice system. Based on those premises, we trained three deep learning architectures, ULMFiT, BERT, and Big Bird, on 612,961 Federal Small Claims Courts appeals within the Brazilian 5th Regional Federal Court to predict their outcomes. We compare the predictive performance of the models to the predictions of 22 highly skilled experts. All models outperform human experts, with the best one achieving a Matthews Correlation Coefficient of 0.3688 compared to 0.1253 from the human experts. Our results demonstrate that natural language processing and machine learning techniques provide a promising approach for predicting legal outcomes. We also release the Brazilian Courts Appeal Dataset for the 5th Regional Federal Court (BrCAD-5), containing data from 765,602 appeals to promote further developments in this area. |
---|---|
Bibliography: | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 Competing Interests: Elias Jacob de Menezes-Neto declares no competing interests. Marco Bruno Miranda Clementino is a federal judge in the 5th Regional Federal Court jurisdiction. Although his position is not affected in any way by this paper, we understand that this affiliation may be seen as a non-financial competing interest. This does not alter our adherence to PLOS ONE policies on sharing data and materials. |
ISSN: | 1932-6203 1932-6203 |
DOI: | 10.1371/journal.pone.0272287 |