Google’s DeepMind AI lab has developed new technology that can generate soundtracks and dialogue to accompany videos. The V2A project combines raw pixel data with text prompts to create sound effects for video content, including traditional silent movies. The system was trained on AI-generated videos, audios, and annotations to learn how to associate sounds with visual scenes.
While other AI tools can also generate sound effects, DeepMind’s technology stands out for its ability to understand raw pixels and optional text input to shape the final product. Users can provide positive or negative cues to influence the sound output. The team is working to address limitations like drop in audio quality from source video distortions and improving lip syncing for generated dialogue.
The researchers are committed to rigorous security evaluations before releasing the technology to the public. This innovation could revolutionize video production by automating the creation of soundtracks and dialogue, enhancing the overall viewing experience. Visit the DeepMind blog to learn more about their V2A technology and its potential applications in the film and entertainment industry.
Article Source
https://www.engadget.com/google-deepminds-new-ai-tech-will-generate-soundtracks-for-videos-113100908.html