Welcome to Runway Academy! Today, we will explore the exciting world of generative audio, which includes text-to-speech, custom voice models, and creating lip sync videos. Generative audio tools can transform the way you work with audio, making it easier to produce high-quality results quickly.
Using the Generative Audio Tool
From your Runway dashboard, click on the generative audio tool. At the top, you’ll find an area where you can type in any text you want and turn it into a spoken audio file.
After writing your text, you can preview and choose a voice from the default voice list. Once you’ve selected a voice, click on the generate button.
Generation times depend on the length of your script but are generally quick. Audio generations are automatically saved to the generative audio folder inside your main assets folder. You can also choose to save it elsewhere using the drop-down menu.
Creating Custom Voice Models
Next, let’s train a custom voice model. You will need a few minutes of clean audio, which you can import into Runway or record directly within the generative audio tool.
You can read from your script or use a provided one, ensuring the audio is as clean as possible.
After recording, name your voice model, and it will be ready to use with text-to-speech in just a few seconds. This feature allows you to create personalized and unique audio outputs tailored to your needs.
Making Lip Sync Videos
Finally, let’s create a lip sync video. To do this, you’ll need an image or video of a person with their full face viewable within the frame. You can upload your own media or use one of the preset characters. For this example, add an image generated in Runway.
Lip sync can be used with generated audio from text-to-speech, recorded audio, or uploaded audio. Choose your voice, click generate, and your lip sync video will be ready. If you want, you can turn your image into a video using Gen 2 and upload it to the generative audio tool to add lip sync.
Note that if your audio is longer than the video, the video will reverse and go back to the beginning once it reaches the end. To avoid the reversing effect, use subject motion with the motion brush instead of camera motion parameters.
Video Tutorial
Conclusion
Thank you for exploring generative audio with us today. For more helpful resources, join Runway community on Discord, where you can find information and experimentation using Runway. Also, Runway has an active community of creators and contributors on Twitter. If you have specific questions, use the help button on your dashboard at any time. Happy creating!