Pitt at CLEF05: Data Fusion for Spoken Document Retrieval

• Published in: Accessing Multilingual Information Repositories pp. 773 - 782
• Main Authors: He, Daqing, Ahn, Jae-Wook
• Format: Book Chapter Conference Proceeding
• Language: English
• Published: Berlin, Heidelberg Springer Berlin Heidelberg 2006; Springer
• Series: Lecture Notes in Computer Science
• Subjects: Applied sciences; Artificial intelligence; Automatic Speech Recognition; Computer science; control theory; systems; Data Fusion; Exact sciences and technology; Information systems. Data bases; Mean Average Precision; Memory organisation. Data processing; Multiple Iteration; Retrieval Result; Software; Speech and sound recognition and synthesis. Linguistics; Document retrieval; Multilingualism; Information system; Speech recognition; Linguistics; Metadata; Data fusion; Data models; Text; Automatic recognition; Modeling
• ISBN: 9783540456971; 354045697X
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/11878773_85

This paper describes an investigation of data fusion techniques for spoken document retrieval. The effectiveness of retrievals solely based on the outputs from automatic speech recognition (ASR) is subject to the recognition errors introduced by the ASR process. This is especially true for retrievals on Malach test collection, whose ASR outputs have average word error rate (WER) of 35%. To overcome the problem, in this year CLEF experiments, we explored data fusion techniques for integrating the manually generated metadata information, which is provided for every Malach document, with the ASR outputs. We concentrated our effort on the post-search data fusion techniques, where multiple retrieval results using automatic generated outputs or human metadata were combined. Our initial studies indicated that a simple unweighted combination method (i.e., CombMNZ) that had demonstrated to be useful in written text retrieval environment only generated significant 38% relative decrease in retrieval effectiveness (measured by Mean Average Precision) for our task by comparing to a simple retrieval baseline where all manual metadata and ASR outputs are put together. This motivated us to explore a more elaborated weighted data fusion model, where the weights are associated with each retrieval result, and can be specified by the user in advance. We also explored multiple iterations of data fusion in our weighted fusion model, and obtained further improvement at 2nd iteration. In total, our best run on data fusion obtained 31% significant relative improvement over the simple fusion baseline, and 4% relative improvement over the manual-only baseline, which is a significant difference.