Transformation Algorithms for Data Streams

Next generation data processing systems must deal with very high data ingest rates and massive volumes of data. Such conditions are typically encountered in the intelligence community (IC) where analysts must search through huge volumes of data in order to gather evidence to support or refute their...

Full description

Saved in:
Bibliographic Details
Published in2005 IEEE Aerospace Conference pp. 1 - 10
Main Authors Eick, S.G., Lockwood, J.W., Loui, R., Moscola, J., Weishar, D.J.
Format Conference Proceeding
LanguageEnglish
Published IEEE 2005
Subjects
Automation
Data processing
Encoding
Hardware
Humans
Information analysis
Information processing
Laboratories
Performance evaluation
System testing
Online AccessGet full text

Cover

More Information
Summary:Next generation data processing systems must deal with very high data ingest rates and massive volumes of data. Such conditions are typically encountered in the intelligence community (IC) where analysts must search through huge volumes of data in order to gather evidence to support or refute their hypotheses. Their effort is made all the more difficult given that the data appears as unstructured text that is written in multiple languages using characters that have different encodings. Human analysts have not been able to keep pace with reading the data and a large amount of data is discarded even though it might contain key information. The goal of our project is to assess the feasibility of incrementally replacing humans with automation in key areas of information processing. These areas include document ingest, content categorization, language translation, and context- and temporally-based information retrieval
ISBN:9780780388703
0780388704
ISSN:1095-323X
2996-2358
DOI:10.1109/AERO.2005.1559611