Using U.S. Census Bureau Documentation to Revisit and Rebuild 20th Century Data Complexities

David Bleckley, University of Michigan
John Sullivan, Census Bureau
Douglas Rosin, University of Michigan

Data and documents are produced within a specific historical context, for a designated purpose, and within an available technological capacity. The interplay of this context, purpose, and capacity informs the decisions producers make regarding what information gets recorded via what media for how long. When future users access, analyze, and interpret data and documentation, their ability to decode, understand, and apply these collections in a new time, for a novel purpose, and with emerging technologies depends upon the producer’s decisions. In this paper, we present the various methods we used to digitize five historical physical data collections and to leverage documentation in these processes as part of the larger Decennial Census Digitization and Linkage project. We begin with a brief description of the methods used to digitize these five datasets from physical collections to machine-actionable datasets, including computer vision-based preprocessing, optical character recognition, and data parsing. The second section of the paper focuses on our use of documentation and data to inform data linkage and application. Specifically, we discuss our use of various documentation files produced during the original production of the data, notably, the decennial census “Procedural Histories.” We describe efforts to trace the original data production process and present examples of how our intended use of these recovered datasets is enabled and constrained by historical documentation. Presenting our experience using documentation of past data collection, tabulation, and storage, this paper seeks to provide insights into the incorporation of documentation in the research process and the production of documentation today for tomorrow’s researchers.

See extended abstract

 Presented in Session 143. Digitizing Analog History