Xuehai He1, Shuohang Wang2, Jianwei Yang2, Xiaoxia Wu2, Yiping Wang3, Kuan Wang4, Zheng Zhan5,
Olatunji Ruwase2, Yelong Shen2, Xin Eric Wang1
1UC Santa Cruz, 2Microsoft, 3University of Washington, 4Georgia Institute of Technology, 5Northeastern University
Motion Directional Control
(a) Text prompt: A serene wide-angle shot captures a sunflower gently swaying in the breeze, set against the backdrop of an expansive golden wheat field under the setting sun.
(b) Text prompt: A red helium balloon floats slowly upward into the sky over a barren desert and expansive Gobi landscape.
(c) Text prompt: A leaf gently drifts down from a tree in a calm forest clearing.
(d) Text prompt: A small red ball bounces along a cobblestone path lined with greenery.
(e) Text prompt: A glowing lantern drifts upwards through the night sky during a lantern festival.
(f) Text prompt: A ring rolls along a marble countertop, catching the light as it moves.
(g) Text prompt: A flickering light bulb swings gently in an empty garage, casting faint shadows on the walls.
(h) Text prompt: A crumpled plastic bag is blown across an empty street by the wind.
Motion Intensity Control
Intensity Level 1
Intensity Level 5
Intensity Level 1
Intensity Level 5
(a) Text prompt: A mojito is poured into a glass continuously from a pitcher, with lime wedges and mint leaves floating around.
(b) Text prompt: A serene underwater scene featuring a sea turtle swimming through a coral reef. The turtle is with its greenish-brown shell.
Intensity Level 1
Intensity Level 10
Intensity Level 1
Intensity Level 10
(c) Text prompt: A mojito sits on a beachside table, with the camera slowly zooming in as the waves crash in the background, ice slightly clinking as it melts.
(d) Text prompt: A helicopter hovers over a snowy mountain peak, with strong winds swirling the snow around and creating intense motion in the sky.
Intensity Level 1
Intensity Level 10
Intensity Level 1
Intensity Level 10
(e) Text prompt: A dramatic thunderstorm over a dense jungle, with flashes of lightning illuminating the towering trees and heavy rain cascading down.
(f) Text prompt: A tranquil beach at sunset, waves gently lapping at the shore, and footprints in the sand disappearing with the tide.
Intensity Level 1
Intensity Level 3
Intensity Level 3
Intensity Level 10
(g) Text prompt: A calm lake at sunrise, where the reflection of the golden sky shimmers on the water, and a lone fisherman casts his line.
(h) Text prompt: A snow-covered cabin in the middle of a dense forest, with smoke curling from its chimney into the cold winter air.
Intensity Level 1
Intensity Level 3
Intensity Level 5
Intensity Level 10
(i) Text prompt: A flickering candle in a dark room, casting gentle light on the walls.
@article{he2024mojito,
title = {Mojito: Motion Trajectory and Intensity Control for Video Generation},
author = {Xuehai He and Shuohang Wang and Jianwei Yang and Xiaoxia Wu and Yiping Wang and Kuan Wang and Zheng Zhan and Olatunji Ruwase and Yelong Shen and Xin Eric Wang},
year = {2024},
journal = {arXiv preprint arXiv: 2412.08948}
}