A Prototype Gutenberg-HathiTrust Sentence-level Parallel Corpus
Citation:
Jiang, Ming; Dubnicek, Ryan; Worthey, Glen; Underwood, Ted; Downie, J. Stephen (2022): A Prototype Gutenberg-HathiTrust Sentence-level Parallel Corpus. University of Illinois at Urbana-Champaign. https://doi.org/10.13012/B2IDB-1685085_V1
Persistent link for this dataset:
Dataset Description

This is a sentence-level parallel corpus in support of research on OCR quality. The source data comes from: (1) Project Gutenberg for human-proofread "clean" sentences; and, (2) HathiTrust Digital Library for the paired sentences with OCR errors. In total, this corpus contains 167,079 sentence pairs from 189 sampled books in four domains (i.e., agriculture, fiction, social science, world war history) published from 1793 to 1984. There are 36,337 sentences that have two OCR views paired with each clean version. In addition to sentence texts, this corpus also provides the location (i.e., sentence and chapter index) of each sentence in its belonging Gutenberg volume.

Subject Social Sciences
Keywords sentence-level parallel corpus; optical character recognition; OCR errors; Project Gutenberg; HathiTrust Digital Library; digital libraries; digital humanities;
License CC BY
Funder HathiTrust Research Center
Corresponding Creator Ming Jiang
Downloaded 86 times
Version DOI Comment Publication Date
1 10.13012/B2IDB-1685085_V1 2022-06-20
Open in Globus File Manager what's this?
513 Bytes File
34.2 MB File
19.9 MB File

Contact the Research Data Service for help interpreting this log.

update: {"version_comment"=>[nil, ""], "subject"=>[nil, "Social Sciences"]} 2022-07-19T19:39:13Z
update: {"publication_state"=>["metadata embargo", "released"]} 2022-06-20T08:00:10Z