A Survey on Video Diffusion Models
Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, Yu-Gang Jiang
(Source: Make-A-Video, SimDA, PYoCo, SVD , Video LDM and Tune-A-Video)
- [News] We are planning to update the survey soon to encompass the latest work. If you have any suggestions, please feel free to contact us.
- [News] The Chinese translation is available on Zhihu. Special thanks to Dai-Wenxun for this.
Open-source Toolboxes and Foundation Models
Methods | Task | Github |
---|---|---|
Open-Sora-Plan | T2V Generation | |
Open-Sora | T2V Generation | |
Morph Studio | T2V Generation | - |
Genie | T2V Generation | - |
Sora | T2V Generation & Editing | - |
VideoPoet | T2V Generation & Editing | - |
Stable Video Diffusion | T2V Generation | |
NeverEnds | T2V Generation | - |
Pika | T2V Generation | - |
EMU-Video | T2V Generation | - |
GEN-2 | T2V Generation & Editing | - |
ModelScope | T2V Generation | |
ZeroScope | T2V Generation | - |
T2V Synthesis Colab | T2V Genetation | |
VideoCraft | T2V Genetation & Editing | |
Diffusers (T2V synthesis) | T2V Genetation | - |
AnimateDiff | Personalized T2V Genetation | |
Text2Video-Zero | T2V Genetation | |
HotShot-XL | T2V Genetation | |
Genmo | T2V Genetation | - |
Fliki | T2V Generation | - |
Table of Contents
Video Generation
Data
Caption-level
Category-level
Title | arXiv | Github | WebSite | Pub. & Date |
---|---|---|---|---|
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild | - | - | Dec., 2012 | |
First Order Motion Model for Image Animation | - | - | May, 2023 | |
Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks | - | - | CVPR,2018 |
Metric and BenchMark
Title | arXiv | Github | WebSite | Pub. & Date |
---|---|---|---|---|
Fréchet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos | - | Jul., 2024 | ||
ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation | Jun., 2024 | |||
[STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative |