What is a Neural Network?, Human Brain, Models of Neuron, Neural Networks viewed as directed graphs, Feedback, Network Architectures, Knowledge representation, Artificial Intelligence and Neural Networks.
Introduction, Error-correction learning, Memory-based learning, Hebbian learning, Competitive learning, Boltzamann learning, Credit Assignment problem, Learning with a Teacher, Learning without a Teacher, Learning tasks, Memory, Adaptation.
Statistical nature of the learning process, Statistical learning theory, Approximately correct model of learning. Single Layer Perceptrons: Introduction, Adaptive filtering problem, Unconstrained optimization techniques, Linear least-squares filters, Least-mean square algorithm, Learning curves, Learning rate annealing techniques, Perceptron, Perceptron convergence theorem, Relation between the Perceptron and Bayes classifier for a Gaussian environment.
Introduction, Some preliminaries, Back-propagation Algorithm, Summary of back-propagation algorithm, XOR problem, Heuristics for making the back-propagation algorithm perform better, Output representation and decision rule, Computer experiment, Feature detection, Back-propagation and differentiation.
Hessian matrix, Generalization, approximation of functions, Cross validation, Network pruning techniques, virtues and limitations of back- propagation learning, Accelerated convergence of back propagation learning, Supervised learning viewed as an optimization problem, Convolution networks.
Introduction, Cover’s theorem on the separability of patterns, Interpolation problem, Supervised learning as an ill-posed Hypersurface reconstruction problem, Regularization theory, Regularization networks, Generalized radial-basis function networks, XOR problem, Estimation of the regularization parameter.
Approximation properties of RBF networks, Comparison of RBF networks and multilayer Perceptrons, Kernel regression and it’s relation to RBF networks, Learning strategies, Computer experiment. Optimization using Hopfield networks: Traveling salesperson problem, Solving simultaneous linear equations, Allocating documents to multiprocessors.
Iterated gradient descent, Simulated Annealing, Random Search, Evolutionary computation- Evolutionary algorithms, Initialization, Termination criterion, Reproduction, Operators, Replacement, Schema theorem.