Flexible character accuracy measure for reading-order-independent evaluation

Clausner, C ORCID: https://orcid.org/0000-0001-6041-1002, Pletschacher, S ORCID: https://orcid.org/0000-0003-0541-0968 and Antonacopoulos, A ORCID: https://orcid.org/0000-0001-9552-0233 2020, 'Flexible character accuracy measure for reading-order-independent evaluation' , Pattern Recognition Letters, 131 , pp. 390-397.

Preview	PDF - Published Version Available under License Creative Commons Attribution 4.0. Download (2MB) | Preview
	PDF - Accepted Version Restricted to Repository staff only Download (1MB) | Request a copy

Abstract

The extraction of textual information from scanned document pages is a fundamental stage in any digitisation effort and directly determines the success of the overall document analysis and understanding application scenarios. To evaluate and improve the performance of optical character recognition (OCR), it is necessary to measure the accuracy of that step alone, without the influence of the processing steps that precede it (e.g. text block segmentation and ordering). Current OCR performance evaluation measures (based on edit distance) are strongly subjective as they need to first serialise the entire text in the documents – a process influenced heavily by the specific reading order determined (often wrongly, especially in cases of multicolumn and complex layouts) by processing steps prior to OCR. This paper presents a new objective and practical edit-distance-based character recognition accuracy measure which overcomes those limitations. It achieves its independence from the reading order by comparing sub-strings of text in a flexible way (i.e. allowing for ordering variations). The precision of the flexible character accuracy measure enables the effective tuning of complete digitisation workflows (as OCR errors are isolated and other steps can be evaluated and optimised separately). For the same reason, it also enables a better estimation of post-OCR (manual) correction effort required. The proposed character accuracy measure has been systematically analysed and validated under lab conditions as well as successfully used in practice in a number of high-profile international competitions since 2017.

Item Type:	Article
Schools:	Schools > School of Computing, Science and Engineering > Salford Innovation Research Centre
Journal or Publication Title:	Pattern Recognition Letters
Publisher:	Elsevier
ISSN:	0167-8655
Related URLs:	Publication
SWORD Depositor:	Publications Router
Depositing User:	USIR Admin
Date Deposited:	07 Feb 2020 13:42
Last Modified:	16 Feb 2022 03:59
URI:	https://usir.salford.ac.uk/id/eprint/56373

Actions (login required)

Edit record (repository staff only)