Text Detection, Tracking and Recognition in Video: A Comprehensive Survey

• Published in: IEEE transactions on image processing Vol. 25; no. 6; pp. 2752 - 2773
• Main Authors: Yin, Xu-Cheng, Zuo, Ze-Yu, Tian, Shu, Liu, Cheng-Lin
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Data mining; Extraction; Feature extraction; Image color analysis; Image detection; Image recognition; Object recognition; Recognition; scene text; Streaming media; Text recognition; Text tracking; Texts; Tracking; tracking based text detection; tracking based text recognition; Transaction processing; Video data; video text extraction; Visualization; tracking based text detection; scene text; tracking based text recognition; Text tracking; video text extraction
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2016.2554321

The intelligent analysis of video data is currently in wide demand because a video is a major source of sensory data in our lives. Text is a prominent and direct source of information in video, while the recent surveys of text detection and recognition in imagery focus mainly on text extraction from scene images. Here, this paper presents a comprehensive survey of text detection, tracking, and recognition in video with three major contributions. First, a generic framework is proposed for video text extraction that uniformly describes detection, tracking, recognition, and their relations and interactions. Second, within this framework, a variety of methods, systems, and evaluation protocols of video text extraction are summarized, compared, and analyzed. Existing text tracking techniques, tracking-based detection and recognition techniques are specifically highlighted. Third, related applications, prominent challenges, and future directions for video text extraction (especially from scene videos and web videos) are also thoroughly discussed.