A novel nonlinear optimization method for fitting a noisy Gaussian activation function

It is significant to fit a Gaussian function with the observation data for artificial intelligence or other engineering fields. Considering the influence of noises, this article proposes a nonlinear optimization method for fitting the Gaussian activation functions. By means of the gradient search and the Newton search, a direct gradient-based iterative algorithm and a direct Newton iterative algorithm are presented for identifying the Gaussian functions. Considering the computational cost, the authors develop a multi-innovation stochastic gradient algorithm for the noisy Gaussian functions. After introducing a forgetting factor, the parameter estimation accuracy can be further improved. The simulation results indicate that the proposed nonlinear optimization method and gradient-based algorithms can fit the noisy Gaussian functions very well.