Lip reading using neural networks

Dhananjay Kalbande; Sanjivani Patil

doi:10.1117/12.913406

30 September 2011 Lip reading using neural networks

Dhananjay Kalbande, Sanjivani Patil

Proceedings Volume 8285, International Conference on Graphic and Image Processing (ICGIP 2011); 828519 (2011) https://doi.org/10.1117/12.913406
Event: 2011 International Conference on Graphic and Image Processing, 2011, Cairo, Egypt

Abstract

Computerized lip reading, or speech reading, is concerned with the difficult task of converting a video signal of a speaking person to written text. It has several applications like teaching deaf and dumb to speak and communicate effectively with the other people, its crime fighting potential and invariance to acoustic environment. We convert the video of the subject speaking vowels into images and then images are further selected manually for processing. However, several factors like fast speech, bad pronunciation, and poor illumination, movement of face, moustaches and beards make lip reading difficult. Contour tracking methods and Template matching are used for the extraction of lips from the face. K Nearest Neighbor algorithm is then used to classify the 'speaking' images and the 'silent' images. The sequence of images is then transformed into segments of utterances. Feature vector is calculated on each frame for all the segments and is stored in the database with properly labeled class. Character recognition is performed using modified KNN algorithm which assigns more weight to nearer neighbors. This paper reports the recognition of vowels using KNN algorithms

Citation Download Citation

Dhananjay Kalbande and Sanjivani Patil "Lip reading using neural networks", Proc. SPIE 8285, International Conference on Graphic and Image Processing (ICGIP 2011), 828519 (30 September 2011); https://doi.org/10.1117/12.913406

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available