The maximum likelihood neural network as a statistical classification model

Neural networks have received considerable attention in recent years. This development has been pursued primarily by non-statisticians. Consequently many statistical tools and concepts have not been utilized in this development and great claims for neural networks have sometimes been made without comparisons to standard statistical procedures. In this paper we utilize the input-output relationship associated with a simple feed-forward neural network as the basis for a non-linear multivariate classifier. A statistical model for the data is defined based on a logistic likelihood function. Neural network parameters are estimated using the method of maximum likelihood instead of the back-propagation technique often used in the neural network literature. An extension for the multinomial case is presented. These maximum likelihood based models can be compared using readily available techniques such as the likelihood ratio test and the Akaike criterion (1973). We provide empirical comparisons of this network approach with standard logistic regression for both the binomial and multinomial cases.