Humanoid robot learns lifelike lip motion from video
Columbia's Creative Machines Lab taught a humanoid robot to produce lifelike lip movements for speaking and singing by self-observing and studying hours of YouTube videos, using a vision-to-action model that maps visuals to synchronized motor commands. The system can synchronize lips across languages and even perform songs, reducing the uncanny-valley effect, though hard consonants like B and W remain challenging. The work, published in Science Robotics, suggests broader implications for education, healthcare, and elder care as facial realism enhances human–robot interaction.