Neural networks: different problems require different learning rate adaptive methods

Remy Allard; Jocelyn Faubert

doi:10.1117/12.527094

28 May 2004 Neural networks: different problems require different learning rate adaptive methods

Remy Allard, Jocelyn Faubert

Proceedings Volume 5298, Image Processing: Algorithms and Systems III; (2004) https://doi.org/10.1117/12.527094
Event: Electronic Imaging 2004, 2004, San Jose, California, United States

Abstract

In a previous study, a new adaptive method (AM) was developed to adjust the learning rate in artificial neural networks: the generalized no-decrease adaptive method (GNDAM). The GNDAM is fundamentally different from other traditional AMs. Instead of using the derivative sign of a given weight to adjust its learning rate, this AM is based on a trial and error heuristic where global learning rates are adjusted according to the error rates produced by two identical networks using different learning rates. This AM was developed to solve a particular task: the orientation detection of an image defined by texture (the texture task). This new task is also fundamentally different from other traditional ones since its data set is infinite, each pattern is a template used to generate stimuli that the network learns to classify. In the previous study, the GNDAM showed its strength over standard backpropagation for this particular task. The present study compares this new AM to other traditional AMs on the texture task and other benchmark tasks. The results showed that some AMs work well for some tasks while others work better for other tasks. However, all of them failed to achieve a good performance on all tasks.

Citation Download Citation

Remy Allard and Jocelyn Faubert "Neural networks: different problems require different learning rate adaptive methods", Proc. SPIE 5298, Image Processing: Algorithms and Systems III, (28 May 2004); https://doi.org/10.1117/12.527094

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available