Insights into the accuracy of social scientists’ forecasts of societal change

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, pol...

Full description

Saved in:
Bibliographic Details
Published inNature human behaviour Vol. 7; no. 4; pp. 484 - 501
Main Authors Grossmann, Igor, Rotella, Amanda, Hutcherson, Cendri A., Sharpinskyi, Konstantyn, Varnum, Michael E. W., Achter, Sebastian, Dhami, Mandeep K., Guo, Xinqi Evie, Kara-Yakoubian, Mane, Mandel, David R., Raes, Louis, Tay, Louis, Vie, Aymeric, Wagner, Lisa, Adamkovic, Matus, Arami, Arash, Arriaga, Patrícia, Bandara, Kasun, Baník, Gabriel, Bartoš, František, Baskin, Ernest, Bergmeir, Christoph, Białek, Michał, Børsting, Caroline K., Browne, Dillon T., Caruso, Eugene M., Chen, Rong, Chie, Bin-Tzong, Chopik, William J., Collins, Robert N., Cong, Chin Wen, Conway, Lucian G., Davis, Matthew, Day, Martin V., Dhaliwal, Nathan A., Durham, Justin D., Dziekan, Martyna, Elbaek, Christian T., Shuman, Eric, Fabrykant, Marharyta, Firat, Mustafa, Fong, Geoffrey T., Frimer, Jeremy A., Gallegos, Jonathan M., Goldberg, Simon B., Gollwitzer, Anton, Goyal, Julia, Graf-Vlachy, Lorenz, Gronlund, Scott D., Hafenbrädl, Sebastian, Hartanto, Andree, Hirshberg, Matthew J., Hornsey, Matthew J., Howe, Piers D. L., Izadi, Anoosha, Jaeger, Bastian, Kačmár, Pavol, Kim, Yeun Joon, Krenzler, Ruslan, Lannin, Daniel G., Lin, Hung-Wen, Lou, Nigel Mantou, Lua, Verity Y. Q., Lukaszewski, Aaron W., Ly, Albert L., Madan, Christopher R., Maier, Maximilian, Majeed, Nadyanna M., March, David S., Marsh, Abigail A., Misiak, Michal, Myrseth, Kristian Ove R., Napan, Jaime M., Nicholas, Jonathan, Nikolopoulos, Konstantinos, O, Jiaqing, Otterbring, Tobias, Paruzel-Czachura, Mariola, Pauer, Shiva, Protzko, John, Raffaelli, Quentin, Ropovik, Ivan, Ross, Robert M., Roth, Yefim, Røysamb, Espen, Schnabel, Landon, Schütz, Astrid, Seifert, Matthias, Sevincer, A. T., Sherman, Garrick T., Simonsson, Otto, Sung, Ming-Chien, Tai, Chung-Ching, Talhelm, Thomas, Teachman, Bethany A., Tetlock, Philip E., Thomakos, Dimitrios, Tse, Dwight C. K., Twardus, Oliver J., Tybur, Joshua M.
Format Journal Article
LanguageEnglish
Published London Nature Publishing Group UK 01.04.2023
Nature Publishing Group
Subjects
4014/4045
4014/477/2811
706/689/477/2811
Accuracy
Averages
Behavioral Sciences
Biomedical and Life Sciences
Experimental Psychology
Forecasting
Humans
Interdisciplinary aspects
Life satisfaction
Life Sciences
Medicin och hälsovetenskap
Microeconomics
Models, Statistical
Neurosciences
Personality and Social Psychology
Polarization
Racial bias
Social media
Social sciences
Teams
Tournaments
Tournaments & championships
Online AccessGet full text

Cover

More Information
Summary:How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N  = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N  = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public ( N  = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data. How accurate are social scientists in predicting societal change, and what processes underlie their predictions? Grossmann et al. report the findings of two forecasting tournaments. Social scientists’ forecasts were on average no more accurate than those of simple statistical models.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
Conceptualization: I.G., A.R., C.A.H., M.E.W.V., L.T. and P.E.T. Data curation: I.G., K.S., G.T.S. and O.J.T. Forecasting: S.A., M.K.D., X.E.G., M. J. Hirshberg, M.K.-Y., D.R.M., L.R., A.V., L.W., M.A., A.A., P.A., K.B., G.B., F.B., E.B., C.B., M.B., C.K.B., D.T.B., E.M.C., R.C., B.-T.C., W.J.C., C.W.C., L.G.C., M. Davis, M.V.D., N.A.D., J.D.D., M. Dziekan, C.T.E., E.S., M. Fabrykant, M. Firat, G.T.F., J.A.F., J.M.G., S.B.G., A.G., J.G., L.G.-V., S.D.G., S.H., A.H., M. J. Hornsey, P.D.L.H., A.I., B.J., P.K., Y.J.K., R.K., D.G.L., H.-W.L., N.M.L., V.Y.Q.L., A.W.L., A.L.L., C.R.M., M. Maier, N.M.M., D.S.M., A.A.M., M. Misiak, K.O.R.M., J.M.N., J.N., K.N., J.O., T.O., M.P.-C., S.P., J.P., Q.R., I.R., R.M.R., Y.R., E.R., L.S., A.S., M.S., A.T.S., O. Simonsson, M.-C.S., C.-C.T., T.T., B.A.T., D.T., D.C.K.T., J.M.T., L.U., D.V., L.V.W., H.A.V., Q.W., K.W., M.E.W., C.E.W., T.Y., K.Y., S.Y., V.R.A., J.R.A.-H., P.A.B., A.B., L.C., M.C., S.D.-H., Z.E.F., C.R.K., S.T.K., A.L.O., L.M., M.S.M., M.F.R.C.M., E.K.M., P.M., J.B.N., W.N., R.B.R., P.S., A.H.S., O. Strijbis, D.S., E.T., A.v.L., J.G.V., M.N.A.W. and T.W. Formal analysis: I.G. and C.A.H. Funding acquisition: I.G. Investigation: I.G., A.R. and C.A.H. Methodology: I.G., A.R., C.A.H., K.S., M.E.W.V., S.A., D.R.M., L.R., L.T., A.V., R.N.C., L.U. and D.V. Project administration: I.G., A.R., M.E.W.V., M.K.-Y. and O.J.T. Resources: I.G., A.R., J.N. and G.T.S. Supervision: I.G. Validation: K.S., X.E.G. and L.W. Visualization: I.G. and M.K.D. Writing—original draft: I.G. Writing—review and editing: I.G., A.R., C.A.H., K.S., M.E.W.V., S.A., M.K.D., X.E.G., M. J. Hirshberg, M.K.-Y., D.R.M., L.R., L.T., A.V., L.W., M.A., A.A., P.A., K.B., G.B., F.B., E.B., C.B., M.B., C.K.B., D.T.B., E.M.C., R.C., B.-T.C., W.J.C., R.N.C., C.W.C., L.G.C., M. Davis, M.V.D., N.A.D., J.D.D., M. Dziekan, C.T.E., E.S., M. Fabrykant, M. Firat, G.T.F., J.A.F., J.M.G., S.B.G., A.G., J.G., L.G.-V., S.D.G., S.H., A.H., M. J. Hornsey, P.D.L.H., A.I., B.J., P.K., Y.J.K., R.K., D.G.L., H.-W.L., N.M.L., V.Y.Q.L., A.W.L., A.L.L., C.R.M., M. Maier, N.M.M., D.S.M., A.A.M., M. Misiak, K.O.R.M., J.M.N., K.N., J.O., T.O., M.P.-C., S.P., J. P., Q.R., I.R., R.M.R., Y.R., E.R., L.S., A.S., M.S., A.T.S., O. Simonsson, M.-C.S., C.-C.T., T.T., B.A.T., P.E.T., D.T., D.C.K.T., J.M.T., L.V.W., H.A.V., Q.W., K. W., M.E.W., C.E.W., T.Y., K.Y. and S.Y.
Author contributions
ISSN:2397-3374
2397-3374
DOI:10.1038/s41562-022-01517-1