English
Edit model card

MotionCtrl Model Card

Introduction

MotionCtrl, when provided with specific camera movements indicated by camera poses or several trajectories, can independently control both the camera and object motion in a generated video. It can be deployed on both Text-to-Video generation models, such as LVDM/VideoCrafter and AnimateDiff, as well as Image-to-Video generation models like SVD.

MotionCtrl + SVD

 

MotionCtrl + LVDM/VideoCrafter

 

MotionCtrl + AnimateDiff

 

Usage

  • Download directly in this repository OR Download with python script
from huggingface_hub import hf_hub_download
motionctrl_lvdm_ckpt = hf_hub_download(repo_id="TencentARC/MotionCtrl", filename="motionctrl.ckpt", repo_type="model")
motionctrl_svd_ckpt = hf_hub_download(repo_id="TencentARC/MotionCtrl", filename="motionctrl_svd.ckpt", repo_type="model")
  • Then generate the controllable video following the instructions in our GitHub repository.

Citation

@inproceedings{wang2023motionctrl,
  title={MotionCtrl: A Unified and Flexible Motion Controller for Video Generation},
  author={Wang, Zhouxia and Yuan, Ziyang and Wang, Xintao and Chen, Tianshui and Xia, Menghan and Luo, Ping and Shan, Yin},
  booktitle={arXiv preprint arXiv:2312.03641},
  year={2023}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model's library. Check the docs .

Spaces using TencentARC/MotionCtrl 4