Cognition and the computational power of connectionist networks

• Published in: Connection science Vol. 12; no. 2; pp. 95 - 110
• Main Author: Hadley, Robert F
• Format: Journal Article
• Language: English
• Published: London Taylor & Francis Group 01.06.2000; Taylor & Francis; Taylor & Francis Ltd
• Subjects: Algorithmics. Computability. Computer arithmetics; Applied sciences; Artificial intelligence; Automata. Abstract machines. Turing machines; Biological and medical sciences; Biological Plausibility; Cognition; Cognition & reasoning; Cognition. Intelligence; Computability; Computer science; Computer science; control theory; systems; Exact sciences and technology; Fundamental and applied biological sciences. Psychology; General aspects; Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects); Neural networks; Philosophical Aspects; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Theoretical computing; Theories; Philosophical aspect; Universal function approximation; Deterministic finite automaton; Simulation; Turing machine; Recurrent network; Deterministic automaton; Finite automaton; Cognition; Neural network; Computability; Biological plausibility
• ISSN: 0954-0091; 1360-0494
• DOI: 10.1080/09540090050129745

This paper examines certain claims of 'cognitive significance' which (wisely or not) have been based upon the theoretical powers of three distinct classes of connectionist networks, namely, the 'universal function approximators', recurrent finite-state simulation networks and Turing equivalent networks. Each class will be considered with respect to its potential in the realm of cognitive modelling. Regarding the first class, I argue that, contrary to the claims of some influential connectionists, feed-forward networks do not possess the theoretical capacity to approximate all functions of interest to cognitive scientists. For example, they cannot approximate many important, recursive (halting) functions which map symbolic strings onto other symbolic strings. By contrast, I argue that a certain class of recurrent networks (i.e. those which closely approximate deterministic finite automata (DFA)) shows considerably greater promise in some domains. However, from a cognitive standpoint, difficulties arise when we consider how the relevant recurrent networks could acquire the weight vectors needed to support DFA simulations. These difficulties are severe in the realm of central high-level cognitive functions. In addition, the class of Turing equivalent networks is here examined. It is argued that the relevance ofsuch networks to cognitive modelling is seriously undermined by their reliance on infinite precision in crucial weights and/or node activations. I also examine what advantages these networks might conceivably possess over and above classical symbolic algorithms. For, from a cognitive standpoint, the Turing equivalent networks present difficulties very similar to certain classical algorithms; they appear highly contrived, their structure is fragile and they exhibit little or no noise tolerance.