A weighted quadratic asymptotic analysis of cost functions used in classifier design with extensions to finite-size training sets

Gerald J. Dobeck

doi:10.1117/12.720614

9 April 2007 A weighted quadratic asymptotic analysis of cost functions used in classifier design with extensions to finite-size training sets

Gerald J. Dobeck

Author Affiliations +

Proceedings Volume 6576, Independent Component Analyses, Wavelets, Unsupervised Nano-Biomimetic Sensors, and Neural Networks V; 657611 (2007) https://doi.org/10.1117/12.720614
Event: Defense and Security Symposium, 2007, Orlando, Florida, United States

Abstract

An analysis of the impact of cost function on classifier design is presented. The well know asymptotic probabilistic approach, that invokes the law of large numbers, is extended by incorporating a piece-wise weighted quadratic approximation. This allows different cost functions to be compared and better quantifies the impact of the cost function on the resulting classifier design. In this paper we show how the choice of several well known cost functions are related to (1) Bayesian optimality, (2) classifier complexity, and (3) the ability to estimate decision boundaries. This work extends previous work that relates classifier design to approximations of the Bayesian posterior probability of class membership (e.g., "Any Reasonable Cost Function Can be Used for A Posteriori Probability Approximation" by M. Saerens, et al., IEEE Transactions on NN, September 2002). Several cost functions are analyzed in the paper including the Lp norm and the maximum mutual information (MMI) criterion. An interesting example that supports the theoretical analysis is presented. For the example the Lp norm (with p=1.1) was shown to successfully estimate the Bayesian optimal class decision boundary while the MMI and the L₂ criteria did not. In addition, a finite-version of the theory is presented that bridges the gap between asymptotic theory and strictly finite-size training sets.

Citation Download Citation

Gerald J. Dobeck "A weighted quadratic asymptotic analysis of cost functions used in classifier design with extensions to finite-size training sets", Proc. SPIE 6576, Independent Component Analyses, Wavelets, Unsupervised Nano-Biomimetic Sensors, and Neural Networks V, 657611 (9 April 2007); https://doi.org/10.1117/12.720614

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
14 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Brain-machine interfaces

Statistical analysis

Inspection

Probability theory

Algorithm development

Bridges

Neural networks

Show All Keywords

Keywords/Phrases

Search In:

Publication Years