Music-to-Dance: EDGE AI generates an infinite number of dance ideas for TikTok based on audio input

From Music to Choreography: How EDGE Generates Its Dances

Even for music samples found in the wild, EDGE can produce high-quality dances using music embeddings from the robust Jukebox model. A frozen Jukebox model is used by EDGE to incorporate input music. A series of five-second dance clips are used to teach a conditional diffusion model how to map the music embedded into the clips. Before joining batches of multiple clips together to create an arbitrary-length full video, temporal constraints are applied to ensure temporal consistency.

Freshly generated music-to-dance outputs

Arbitrary temporal and spatial limitations are supported by EDGE. There are numerous end-user applications that can be supported by this, including:

• By requiring temporal continuity across batches of numerous sequences, arbitrary-length dances can be produced.
• Dances that include joint restrictions, such as lower body motion causing upper body generation, or the opposite.
• Dances with beginning and ending moves that are predetermined.
• Dances that begin with a predetermined motion are known as dance continuations.

Researchers found that human observers overwhelmingly preferred dances generated by the EDGE algorithm to those generated by other algorithms. The study’s authors say that the EDGE algorithm represents a major advance in the field of computer-generated dance. They believe that the algorithm could be used to create dances for real-world and virtual events, such as social and TikTok trends, weddings, parties, or corporate functions.

Read more about AI: