The RunwayML Team holds the belief that the next significant breakthrough in AI will stem from systems capable of comprehending the visual world and its dynamics. This conviction has led them to initiate a new, long-term research project focused on what they term general world models.
A world model, in their view, is an AI system that constructs an internal representation of an environment and utilizes this representation to predict future occurrences within that environment. Until now, research in world models has predominantly concentrated on highly restricted and controlled scenarios, such as simulated environments akin to video games or specific contexts like the development of world models for driving. The objective of general world models, as envisioned by the Runway Team, is to depict and simulate a broad spectrum of situations and interactions, mirroring those in the real world.
The team regards video generative systems like Runway Gen-2 as rudimentary and constrained precursors to general world models. For Gen-2 to create convincing short videos, it has had to acquire a basic grasp of physics and motion. Nevertheless, its capabilities remain significantly limited, particularly in handling complex camera or object movements, among other aspects.
In their quest to develop general world models, the Runway Team faces multiple ongoing research challenges. These models must be capable of generating consistent environmental maps and possess the ability to navigate and interact within these environments. They should not only capture the dynamics of the world but also the dynamics of its inhabitants, which includes the creation of realistic human behavior models.
The team is actively assembling a group of experts to address these challenges. They are open to and enthusiastic about hearing from individuals interested in contributing to this research endeavor.
Read other articles: