Visual speech generator

Gregor A. Kalberer; Pascal Mueller; Luc J. Van Gool

doi:10.1117/12.473124

10 January 2003 A Visual Speech Generator

Gregor A. Kalberer, Pascal Mueller, Luc J. Van Gool

Proceedings Volume 5013, Videometrics VII; (2003) https://doi.org/10.1117/12.473124
Event: Electronic Imaging 2003, 2003, Santa Clara, CA, United States

Abstract

Efficient, realistic face animation is still a challenge. A system is proposed that yields realistic animations for speech. It starts from real 3D face dynamics, observed at a frame rate of 25 fps for thousands of points on the faces of speaking actors. When asked to animate a face it replicates the visemes that is has learned, and adds the necessary coarticulation effects. The speech animation could be based on as few as 16 modes, extracted through Independent Component Analysis from the observed face dynamics. Also faces for which only a static, neutral 3D model is available, can be animated. Rather then animating via verbatim copying other faces’ deformation fields, the visemes are adapted to the shape of the new face. By localising this face in a Face Space, where also the locations of the example faces are known, visemes are adapted automatically according to the relative distance with respect to these examples. The animation tool proposes a good speech-based face animation as a point of departure for animators, who also get support by the system to then make further changes as desired.

Citation Download Citation

Gregor A. Kalberer, Pascal Mueller, and Luc J. Van Gool "A Visual Speech Generator", Proc. SPIE 5013, Videometrics VII, (10 January 2003); https://doi.org/10.1117/12.473124

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available