1、项目介绍：

2、环境安装

  • 平台：windows 10
  • 编译器：pycharm
  • cuda 11.3
  • cudnn 8.2.0.53
conda create -n  text2video python=3.8
conda activate text2video
pip install torch-1.11.0+cu113-cp38-cp38-win_amd64.whl
pip install torchvision-0.12.0+cu113-cp38-cp38-win_amd64.whl
pip install modelscope==1.4.2 -i https://mirror.baidu.com/pypi/simple
pip install open_clip_torch -i https://mirror.baidu.com/pypi/simple
pip install pytorch-lightning -i https://mirror.baidu.com/pypi/simple
pip install opencv-python -i https://mirror.baidu.com/pypi/simple 
pip install tensorflow==2.8.0 -i https://mirror.baidu.com/pypi/simple

3、执行代码

from modelscope.pipelines import pipeline
from modelscope.outputs import OutputKeys

p = pipeline('text-to-video-synthesis', 'damo/text-to-video-synthesis')
test_text = {
        'text': 'A panda eating bamboo on a rock.',
    }
output_video_path = p(test_text,)[OutputKeys.OUTPUT_VIDEO]
print('output_video_path:', output_video_path)

4、效果展示

5、总结

Text-to-video-synthesis Model in Open Domain生成的视频基本符合描述，缺点就是画质水平还比较低