Generic architecture for indexing document groups in an inverted text index

A method for indexing a plurality of documents, that includes a plurality of duplicate documents, first identifies one or more duplicate groups of documents from among the plurality of documents. Then, one index of content for the duplicate group is created instead of indexing the content from every...

Full description

Saved in:
Bibliographic Details
Main Authors Broder, Andrei Z, Fontoura, Marcus Felipe, Herscovici, Michael, Lempel, Ronny, McPherson, Jr, John Ai, Neumann, Andreas, Qi, Runping, Shekita, Eugene Jon
Format Patent
LanguageEnglish
Published 06.03.2012
Online AccessGet full text

Cover

More Information
Summary:A method for indexing a plurality of documents, that includes a plurality of duplicate documents, first identifies one or more duplicate groups of documents from among the plurality of documents. Then, one index of content for the duplicate group is created instead of indexing the content from every document within the duplicate group. However, in contrast to the content index, an index of metadata for each of the documents in the duplicate group is created. Thus the content of each duplicate group is indexed only once, while a search engine using such indexing techniques retains the capability to answer queries as if the duplicated content was indexed for each document of the group.