Non-Parametric Detection of the Number of Signals: Hypothesis Testing and Random Matrix Theory

• Published in: IEEE transactions on signal processing Vol. 57; no. 10; pp. 3930 - 3941
• Main Authors: Kritchman, S., Nadler, B.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.10.2009; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Algorithms; Applied sciences; Array signal processing; Arrays; Consistency; Detection; Estimators; Exact sciences and technology; Information, signal and communications theory; Matrix theory; Miscellaneous; number of signals; Optimization; Probability; random matrix theory; Sensor arrays; Signal analysis; Signal detection; Signal processing; Signal processing algorithms; Statistical analysis; statistical hypothesis tests; Telecommunications and information theory; Testing; Tracy-Widom distribution; Performance evaluation; number of signals; Random matrix; Array signal processing; Optimal test; Optimal detection; Algorithm; statistical hypothesis tests; Statistical method; Hypothesis test; Statistical test; Simulation; MDL criterion; Signal processing; random matrix theory; Tracy-Widom distribution; Detection
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2009.2022897

Detection of the number of signals embedded in noise is a fundamental problem in signal and array processing. This paper focuses on the non-parametric setting where no knowledge of the array manifold is assumed. First, we present a detailed statistical analysis of this problem, including an analysis of the signal strength required for detection with high probability, and the form of the optimal detection test under certain conditions where such a test exists. Second, combining this analysis with recent results from random matrix theory, we present a new algorithm for detection of the number of sources via a sequence of hypothesis tests. We theoretically analyze the consistency and detection performance of the proposed algorithm, showing its superiority compared to the standard minimum description length (MDL)-based estimator. A series of simulations confirm our theoretical analysis.