The Costs and Benefits of Incorporating Artificial Intelligence into the Global and Historical Sociology Toolkit

Beverly Silver, Johns Hopkins University
Haohang Guo, Johns Hopkins University
Corey R. Payne, University of Richmond

Big questions about epochal social transformations motivated many of the great works in sociology since the founding of the discipline. But the insufficient temporal and spatial scope of existing datasets have too frequently discouraged others from following in their footsteps. One solution has been to construct new datasets from scratch that are specifically designed for the study of long-term transnational or global processes. The large time commitment needed to bring these projects to fruition—both the large front-end commitment to resolving questions of conceptualization and measurement as well as the data collection process itself—is a reason why many researchers decide not to pursue this path, even when convinced it is the best path forward from a scientific viewpoint. Given recent advances, would integrating AI tools make these projects more feasible? With this question in mind, this paper reports on findings from pilot studies (funded by JHU’s IDIES and DSAI) in which we experimented with natural language processing (NLP), artificial intelligence (AI) and large language models (LLM) as aids for constructing the Global Social Protest Dataset at the Arrighi Center for Global Studies—a new dataset on social protest worldwide from 1792-2023. Our main takeaway from these pilot studies is that there are costs and benefits to integrating AI into the social science history toolkit, and researchers need to be conscious of both. On the benefits side of the ledger, with big caveats, AI can increase the speed/efficiency of data collection, annotation and classification. Among the costs are the difficulty of training and deploying AIs on historical texts (creating incentives to limit research to the contemporary period), the black-box of AI “decision-making”, the risk of sidelining theory, and the ways in which AI (broadly defined) transforms human collaborations within the research process including pedagogy/mentoring and attribution of human scholarship.

No extended abstract or paper available

 Presented in Session 107. AI Impact on Data Infrastructure