Runway Gen-3 Alpha marks the beginning of a new series of models developed by Runway, utilizing a newly constructed infrastructure designed for large-scale multimodal training. This model represents a significant enhancement in fidelity, consistency, and motion compared to Gen-2, and it is a key advancement towards creating General World Models.
Gen-3 Alpha has been trained using highly descriptive, temporally dense captions, allowing for imaginative transitions and precise key-framing of elements within a scene.
Gen-3 Alpha excels at creating expressive human characters capable of a wide range of actions, gestures, and emotions, thus unlocking new storytelling opportunities.
Industry Customization
As part of the Gen-3 model family, Runway Dev team collaborated with leading entertainment and media organizations to develop custom versions of Gen-3. This customization enables more stylistically controlled and consistent characters, tailored to meet specific artistic and narrative requirements, among other features.
Trained jointly on videos and images, Gen-3 Alpha will power Runway’s Text to Video, Image to Video and Text to Image tools, existing control modes such as Motion Brush, Advanced Camera Controls and Director Mode, and upcoming tools to enable even more fine-grained control over structure, style and motion. Gen-3 Alpha will also be released with a new set of safeguards, including a new and improved in-house visual moderation system and C2PA provenance standards.
Gen-3 Alpha was trained from the ground up for creative applications. It was a collaborative effort from a cross-disciplinary team of research scientists, engineers and artists.
As part of the family of Gen-3 models, we have been collaborating and partnering with leading entertainment and media organizations to create custom versions of Gen-3 Alpha. Customization of Gen-3 models allows for even more stylistically controlled and consistent characters, and targets specific artistic and narrative requirements.
This leap forward in technology represents a significant milestone in our commitment to empowering artists, paving the way for the next generation of creative and artistic innovation. Gen-3 Alpha will be available for everyone over the coming days.
Prompt: A middle-aged sad bald man becomes happy as a wig of curly hair and sunglasses fall suddenly on his head.
What is General World Model?
A world model is an AI system that constructs an internal representation of an environment and uses it to simulate future events within that environment. Historically, research in world models has focused on highly limited and controlled settings, such as toy simulated worlds (like those in video games) or narrow contexts (such as developing models for driving). The goal of general world models is to represent and simulate a wide range of situations and interactions, similar to those encountered in the real world.
Video generative systems like Gen-2 can be considered very early and limited forms of general world models. To generate realistic short videos, Gen-2 has developed a basic understanding of physics and motion. However, it still has significant limitations, particularly with complex camera or object motions, among other challenges.
To develop general world models, several open research challenges need to be addressed. These models must generate consistent maps of the environment and be capable of navigating and interacting within those environments. They need to capture not only the dynamics of the world but also the dynamics of its inhabitants, which involves building realistic models of human behavior.
Read related articles: