A Prototype Gutenberg-HathiTrust Sentence-level Parallel Corpus
Dataset Description |
This is a sentence-level parallel corpus in support of research on OCR quality. The source data comes from: (1) Project Gutenberg for human-proofread "clean" sentences; and, (2) HathiTrust Digital Library for the paired sentences with OCR errors. In total, this corpus contains 167,079 sentence pairs from 189 sampled books in four domains (i.e., agriculture, fiction, social science, world war history) published from 1793 to 1984. There are 36,337 sentences that have two OCR views paired with each clean version. In addition to sentence texts, this corpus also provides the location (i.e., sentence and chapter index) of each sentence in its belonging Gutenberg volume.
|
Subject |
Social Sciences |
Keywords |
sentence-level parallel corpus; optical character recognition; OCR errors; Project Gutenberg; HathiTrust Digital Library; digital libraries; digital humanities; |
License |
CC BY |
Funder |
HathiTrust Research Center |
Corresponding Creator |
Ming Jiang |
Downloaded |
489 times |
| Version | DOI | Comment | Publication Date |
|---|---|---|---|
| 1 | 10.13012/B2IDB-1685085_V1 | 2022-06-20 |
Contact the Research Data Service for help interpreting this log.
| RelatedMaterial | destroy: {"material_type"=>"Article", "availability"=>nil, "link"=>"", "uri"=>"", "uri_type"=>"", "citation"=>"in press", "dataset_id"=>2252, "selected_type"=>"Article", "datacite_list"=>"", "note"=>"", "feature"=>nil} | 2025-01-08T23:48:46Z |
| RelatedMaterial | create: {"material_type"=>"Preprint", "availability"=>nil, "link"=>"https://doi.org/10.48550/arXiv.2409.19735", "uri"=>"10.48550/arXiv.2409.19735", "uri_type"=>"DOI", "citation"=>"Bourne, J. (2024). Scrambled text: training Language Models to correct OCR errors using synthetic data. ArXiv, abs/2409.19735.", "dataset_id"=>2252, "selected_type"=>"Other", "datacite_list"=>"IsCitedBy", "note"=>nil, "feature"=>nil} | 2024-10-23T16:28:41Z |
| RelatedMaterial | update: {"note"=>[nil, ""]} | 2024-10-23T16:28:41Z |
| RelatedMaterial | update: {"uri"=>[nil, ""], "uri_type"=>[nil, ""], "datacite_list"=>[nil, ""]} | 2022-07-19T19:39:13Z |
| Dataset | update: {"version_comment"=>[nil, ""], "subject"=>[nil, "Social Sciences"]} | 2022-07-19T19:39:13Z |
| Dataset | update: {"publication_state"=>["metadata embargo", "released"]} | 2022-06-20T08:00:10Z |