Skip to page content
USDA Forest Service
  
Treesearch

Research & Development Treesearch

 
Treesearch Home
About Treesearch
Contact Us
Research & Development
Forest Products Lab
International Institute of Tropical Forestry
Northern
Pacific Northwest
Pacific Southwest
Rocky Mountain
Southern Research Station
Help
 

GeoTreesearch


Science.gov - We Participate


USA.gov  Government Made Easy


Global Forest Information Service

US Forest Service
P.O. Box 96090
Washington, D.C.
20090-6090

(202) 205-8333

You are here: Home / Search / Publication Information
Bookmark and Share

Publication Information

(65 KB bytes)

Title: Fast Back-Propagation Learning Using Steep Activation Functions and Automatic Weight

Author: Cho, Tai-Hoon; Conners, Richard W.; Araman, Philip A.

Date: 1992

Source: Proceedings, 1991 IEEE International Conference on Systems, Man, and Cybernetics. pp. 1587-1592.

Publication Series: Miscellaneous Publication

Description: In this paper, several back-propagation (BP) learning speed-up algorithms that employ the ãgainä parameter, i.e., steepness of the activation function, are examined. Simulations will show that increasing the gain seemingly increases the speed of convergence and that these algorithms can converge faster than the standard BP learning algorithm on some problems. However, these algorithms may also suffer from increased instability, i.e., they frequently fail to converge within a finite time. One potential cause for the instability is an inappropriate choice for the initial weights. To overcome the instability resulting from this cause it is proposed that automatic weight reinitialization be used whenever the convergence speed becomes ãvery slowä due to a local minimum or premature saturation. On the simulations performed BP algorithms with larger initial gain (around 2 or 3) and automatic weight reinitialization converged much faster and were more stable than algorithms employing the same gain but not employing automatic weight reinitialization. The simulations performed involved a diverse set of problems including exclusive-or (XOR), encoder, and parity problems.

Publication Notes:

  • We recommend that you also print this page and attach it to the printout of the article, to retain the full citation information.
  • This article was written and prepared by U.S. Government employees on official time, and is therefore in the public domain.
  • You may send email to pubrequest@fs.fed.us to request a hard copy of this publication. (Please specify exactly which publication you are requesting and your mailing address.)

XML: View XML

Citation:


Cho, Tai-Hoon; Conners, Richard W.; Araman, Philip A. 1992. Fast Back-Propagation Learning Using Steep Activation Functions and Automatic Weight. Proceedings, 1991 IEEE International Conference on Systems, Man, and Cybernetics. pp. 1587-1592.

 


 [ Get Acrobat ]  Get the latest version of the Adobe Acrobat reader or Acrobat Reader for Windows with Search and Accessibility

USDA logo which links to the department's national site. Forest Service logo which links to the agency's national site.