What’s New? What’s Certain? – Scoring Search Results in the Presence of Overlapping Data Sources

• Published in: Data Integration in the Life Sciences pp. 231 - 246
• Main Authors: Hussels, Philipp, Trißl, Silke, Leser, Ulf
• Format: Book Chapter
• Language: English
• Published: Berlin, Heidelberg Springer Berlin Heidelberg 2007
• Series: Lecture Notes in Computer Science
• ISBN: 3540732543; 9783540732549
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-540-73255-6_19

Data integration projects in the life sciences often gather data on a particular subject from multiple sources. Some of these sources overlap to a certain degree. Therefore, integrated search results may be supported by one, few, or all data sources. To reflect these differences, results should be ranked according to the number of data sources that support them. How such a ranking should look like is not clear per se. Either, results supported by only few sources are ranked high because this information is potentially new, or such results are ranked low because the strength of evidence supporting them is limited. We present two scoring schemes to rank search results in the integrated protein annotation database Columba. We define a surprisingness score, preferring results supported by few sources, and a confidence score, preferring frequently encountered information. Unlike many other scoring schemes our proposal is purely data-driven and does not require users to specify preferences among sources. Both scores take the concrete overlaps of data sources into account and do not presume statistical independence. We show how our schemes have been implemented efficiently using SQL.