Minimum fuel neural networks and their applications to overcomplete signal representations

• Published in: IEEE transactions on circuits and systems. 1, Fundamental theory and applications Vol. 47; no. 8; pp. 1146 - 1159
• Main Authors: Wang, Z.S., Cheung, J.Y., Xia, Y.S., Chen, J.D.Z.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2000; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Constraint optimization; Convergence; Dictionaries; Fuels; Matching pursuit algorithms; Neural networks; Optimization methods; Pursuit algorithms; Signal representations; Studies; Time frequency analysis
• ISSN: 1057-7122; 1558-1268
• DOI: 10.1109/81.873870

The overcomplete signal representation (OSR) is a recently established adaptive signal representation method. As an adaptive signal representation method, the OSR means that a given signal is decomposed onto a number of optimal basis components, which are found from an overcomplete basis dictionary via some optimization algorithms, such as the matching pursuit (MP), method of frame (MOF) and basis pursuit (BP). Such ideas are actually very close to or exactly the same as solving a minimum fuel (MF) problem. The MF problem is a well-established minimum L/sub 1/-norm optimization model with linear constraints. The BP-based OSR proposed by Chen and Donoho is exactly the same model as the MF model. The work of Chen and Donoho showed that the MF model could be used as a generalized method for solving an OSR problem and it outperformed the MP and the MOF. In this paper, the neural implementation of the MF model and its applications to the OSR are presented. A new neural network, namely the minimum fuel neural network (MFNN), is constructed and its convergence in solving the MP problem is proven theoretically and validated experimentally. Compared with the implementation of the original BP, the MFNN does not double the scales of the problem and its convergence is independent of initial conditions. It is shown that the MFNN is promising for the application in the OSR's of various kinds of nonstationary signals with a high time-frequency resolution and feasibility of real-time implementation. As an extension, a two-dimensional (2-D) MF model suitable for image data compression is also proposed and its neural implementation is presented.