Robust speech separation using visually constructed speech signals

Parham Aarabi; Negar Habibi Khameneh

doi:10.1117/12.458389

6 March 2002 Robust speech separation using visually constructed speech signals

Parham Aarabi, Negar Habibi Khameneh

Proceedings Volume 4731, Sensor Fusion: Architectures, Algorithms, and Applications VI; (2002) https://doi.org/10.1117/12.458389
Event: AeroSense 2002, 2002, Orlando, FL, United States

Abstract

A technique to virtually recreate speech signals entirely from the visual lip motions of a speaker is proposed. By using six geometric parameters of the lips obtained from the Tulips1 database, a virtual speech signal is recreated by using a 3.6s audiovisual training segment as a basis for the recreation. It is shown that the virtual speech signal has an envelope that is directly related to the envelope of the original acoustic signal. This visual signal envelope reconstruction is then used as a basis for robust speech separation where all the visual parameters of the different speakers are available. It is shown that, unlike previous signal separation techniques, which required an ideal mixture of independent signals, the mixture coefficients can be very accurately estimated using the proposed technique in even non-ideal situations.

Citation Download Citation

Parham Aarabi and Negar Habibi Khameneh "Robust speech separation using visually constructed speech signals", Proc. SPIE 4731, Sensor Fusion: Architectures, Algorithms, and Applications VI, (6 March 2002); https://doi.org/10.1117/12.458389

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available