摘要：我们将给猫贴一张卡通脸，给 Elon Musk 贴上小胡子，给小狗贴上驯鹿角！
本文分享自华为云社区《视频AI，给你的宠物加个表情特效！》，作者：HWCloudAI。
GAN 监督学习是一种联合端到端学习判别模型及其 GAN 生成的训练数据的方法。GANgealing将框架应用于密集视觉对齐问题。受经典 Congealing 方法的启发，GANgealing 算法训练空间变换器将随机样本从在未对齐数据上训练的 GAN 扭曲为共同的、联合学习的目标模式。目标模式已更新，以使空间转换器的工作“尽可能简单”。Spatial Transformer 专门针对 GAN 图像进行训练，并在测试时自动推广到真实图像。
我们可以使用它来进行密集跟踪或创建物镜。例如，我们将给猫贴一张卡通脸，给 Elon Musk 贴上小胡子，给小狗贴上驯鹿角！
实验步骤1.安装依赖包
安装完成之后需要重启Kernel，重启之后才会加载新安装的PyTorch库
!export CXX=g++!pip install ninja==1.11.1 ray==2.1.0 plotly==4.14.3 torch==1.10.1 torchvision==0.11.2 moviepy==0.2.3.5 lmdb==0.99
2.下载代码
import osimport moxing as moxif not os.path.exists('gangealing/'): mox.file.copy_parallel('obs://weilin/gangealing/', 'gangealing/')
3.进入案例文件夹
cd gangealing/gangealing
model：要检测的物体，celeba 代表人、dog代表狗、 cat代表猫、 cub代表鸟
pic：要添加的特效图片
video_name：要添加特效的视频
model = 'cat' #@param ['celeba', 'dog', 'cat', 'cub']pic = 'ModelArts.png'video_name = 'demo.mp4'os.environ['RAW_VIDEO_PATH'] = video_name!chmod 777 ./ffmpegos.environ['FFMPEG_BINARY'] = os.path.join(os.getcwd(), 'ffmpeg')
4.对视频进行抽帧
from pathlib import Pathfrom utils.download import download_model, download_videofrom applications.mixed_reality import run_gangealing_on_videofrom applications import load_stnfrom glob import globvideo_resolution = "512" #@param [128, 256, 512, 1024, 2048, 4096, 8192]pad_mode = 'center' #@param ["center", "border"]os.environ['FFMPEG_BINARY'] = os.path.join(os.getcwd(), 'ffmpeg')os.environ['VIDEO_SIZE'] = video_size = str(video_resolution)os.environ['PAD'] = pad_modevideo = Path(os.environ['RAW_VIDEO_PATH']).stemos.environ['FRAME_PATH'] = f'data/video_frames/{video}'os.environ['VIDEO_NAME'] = videovideo_path = f'data/{video}'!chmod 777 process_video.sh!./process_video.sh "$RAW_VIDEO_PATH"!python prepare_data.py --path "$FRAME_PATH" --out "data/$VIDEO_NAME" --pad "$PAD" --size "$VIDEO_SIZE"
5.为视频添加特效
根据视频的长度和硬件规格，运行此单元需要几分钟，您可以在下方监控进度。
fps = 30batch_size = 1use_flipping = Falsememory_efficient_but_slower = Falseif 'cutecat' in video_path: fps = 60class MyDict(): def __init__(self): passargs = MyDict()args.real_size = int(video_size)args.real_data_path = video_pathargs.fps = fpsargs.batch = batch_sizeargs.transform = ['similarity', 'flow']args.flow_size = 128args.stn_channel_multiplier = 0.5args.num_heads = 1args.distributed = False # Colab only uses 1 GPUargs.clustering = Falseargs.cluster = Noneargs.objects = Trueargs.no_flip_inference = not use_flippingargs.save_frames = memory_efficient_but_slowerargs.overlay_congealed = Falseargs.ckpt = modelargs.override = Falseargs.out = 'visuals'if pic == 'dense tracking': args.label_path = f'assets/masks/{model}_mask.png' # Feel free to change the parameters below: args.resolution = 128 args.sigma = 1.3 args.opacity = 0.8 args.objects = Falseelse: # object lense args.label_path = f'assets/objects/{model}/{pic}' args.resolution = 4 * int(video_size) args.sigma = 0.3 args.opacity = 1.0 args.objects = Truestn = load_stn(args)print('Running Spatial Transformer on frames...')run_gangealing_on_video(args, stn, classifier=None)print('Preparing videos to be displayed...')from IPython.display import HTMLfrom base64 import b64encodenum = len(list(glob(f'{video}_compressed*')))compressed_name = f'{video}_compressed{num}.mp4'congealed_compressed_name = f'{video}_compressed_congealed{num}.mp4'path = f'visuals/video_{video}/propagated.mp4'congealed_path = f'visuals/video_{video}/congealed.mp4'os.system(f"ffmpeg -i {path} -vcodec libx264 {compressed_name}")os.system(f"ffmpeg -i {congealed_path} -vcodec libx264 {congealed_compressed_name}")
6.添加特效前的视频
mp4 = open(video_name,'rb').read()data_url = "data:video/mp4;base64," + b64encode(mp4).decode()HTML(""""%s" type="video/mp4">""" % (data_url))
7.添加特效后的视频
mp4_1 = open(compressed_name,'rb').read()data_url_1 = "data:video/mp4;base64," + b64encode(mp4_1).decode()HTML(""""%s" type="video/mp4">""" % (data_url_1))
8.制作自己的特效视频
上传自己的视频，将视频放在gangealing/gangealing/下面
上传自己的图片，将图片放在gangealing/gangealing/assets/objects/*/对应的种类的文件夹下面，自己制作的特效图片尺寸要是8192×8192
修改步骤3里的3个参数，重新运行一遍即可！
点击关注，第一时间了解华为云新鲜技术~