Audio-visual

How Does it Sound? Generation of Rhythmic Soundtracks for Human Movement Videos (NeurIPS 2021)

A novel system that gets as an input video frames of a human motion and generates the soundtrack for that video.

Audeo: Audio Generation for a Silent Performance Video (NeurIPS 2020)

A novel system that gets as an input video frames of a musician playing the piano and generates the music for that video.