We gratefully acknowledge support from the Simons Foundation, member institutions , and all contributors. Donate [Submitted on 6 Aug 2021]

Title: Lights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents

View PDF Abstract: Document digitization is essential for the digital transformation of our societies, yet a crucial step in the process, Optical Character Recognition (OCR), is still not perfect. Even commercial OCR systems can produce questionable output depending on the fidelity of the scanned documents. In this paper, we demonstrate an effective framework for mitigating OCR errors for any downstream NLP task, using Named Entity Recognition (NER) as an example. We first address the data scarcity problem for model training by constructing a document synthesis pipeline, generating realistic but degraded data with NER labels. We measure the NER accuracy drop at various degradation levels and show that a text restoration model, trained on the degraded data, significantly closes the NER accuracy gaps caused by OCR errors, including on an out-of-domain dataset. For the benefit of the community, we have made the document synthesis pipeline available as an open-source project. Comments: Accepted to the Document Intelligence Workshop at KDD 2021. The source code of Genalog is available at this https URL Subjects: Computation and Language (cs.CL) ; Machine Learning (cs.LG) Cite as: arXiv:2108.02899 [cs.CL] arXiv:2108.02899v1 [cs.CL] for this version) View a PDF of the paper titled Lights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents, by Amit Gupte and 8 other authors

View PDF

TeX Source

Other Formats

view license Current browse context: recent | 2021-08 Change to browse by: cs.LG

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .