pytorch官网

https://pytorch.org/tutorials/advanced/neural_style_tutorial.html

原理

风格迁移的原理基于卷积神经网络的特征提取能力。具体来说，给定一张内容图像和一张风格图像，我们可以将这两张图像通过一个预训练好的卷积神经网络，比如VGG网络，分别传入网络中并提取它们的特征表示。然后，我们可以用这些特征表示来计算新图像的内容和风格表示，再将这些表示结合起来生成新的图像。

具体而言，我们可以通过最小化以下损失函数来生成新的图像：

L t o t a l = α L c o n t e n t + β L s t y l e L_{total} = \alpha L_{content} + \beta L_{style} Ltotal=αLcontent+βLstyle

其中， L c o n t e n t L_{content} Lcontent是内容损失，它衡量新图像与原始内容图像的相似程度， L s t y l e L_{style} Lstyle是风格损失，它衡量新图像与原始风格图像的相似程度， α \alpha α和 β \beta β分别是内容和风格的权重参数。

内容损失通常使用均方误差（MSE）来定义，它测量新图像与原始内容图像在神经网络中某一层的特征表示之间的差异。风格损失则是基于Gram矩阵计算的，它测量新图像与原始风格图像在神经网络中多层特征表示之间的相关性。

通过调整 α \alpha α和 β \beta β的权重，我们可以控制新图像的内容和风格之间的比例，从而生成具有不同特征的图像。

实现

一些库已经有实现，比如Deep Image Analogy。

pytorch的代码：

from __future__ import print_functionimport torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optimfrom PIL import Image
import matplotlib.pyplot as pltimport torchvision.transforms as transforms
import torchvision.models as modelsdevice = torch.device("cuda" if torch.cuda.is_available() else "cpu")# desired size of the output image
imsize = 512 if torch.cuda.is_available() else 128  # use small size if no gpuloader = transforms.Compose([transforms.Resize((imsize, imsize)),  # scale imported imagetransforms.ToTensor()])  # transform it into a torch tensordef image_loader(image_name):image = Image.open(image_name)# fake batch dimension required to fit network's input dimensionsimage = loader(image).unsqueeze(0)return image.to(device, torch.float)style_img = image_loader("style.jpeg")
content_img = image_loader("content2.jpg")assert style_img.size() == content_img.size(), \"we need to import style and content images of the same size"unloader = transforms.ToPILImage()  # reconvert into PIL imageplt.ion()def imshow(tensor, title=None):image = tensor.cpu().clone()  # we clone the tensor to not do changes on itimage = image.squeeze(0)  # remove the fake batch dimensionimage = unloader(image)plt.imshow(image)if title is not None:plt.title(title)plt.pause(0.001)  # pause a bit so that plots are updatedplt.figure()
imshow(style_img, title='Style Image')plt.figure()
imshow(content_img, title='Content Image')class ContentLoss(nn.Module):def __init__(self, target, ):super(ContentLoss, self).__init__()# we 'detach' the target content from the tree used# to dynamically compute the gradient: this is a stated value,# not a variable. Otherwise the forward method of the criterion# will throw an error.self.target = target.detach()def forward(self, input):self.loss = F.mse_loss(input, self.target)return inputdef gram_matrix(input):a, b, c, d = input.size()  # a=batch size(=1)# b=number of feature maps# (c,d)=dimensions of a f. map (N=c*d)features = input.view(a * b, c * d)  # resise F_XL into \hat F_XLG = torch.mm(features, features.t())  # compute the gram product# we 'normalize' the values of the gram matrix# by dividing by the number of element in each feature maps.return G.div(a * b * c * d)class StyleLoss(nn.Module):def __init__(self, target_feature):super(StyleLoss, self).__init__()self.target = gram_matrix(target_feature).detach()def forward(self, input):G = gram_matrix(input)self.loss = F.mse_loss(G, self.target)return inputcnn = models.vgg19(pretrained=True).features.to(device).eval()cnn_normalization_mean = torch.tensor([0.485, 0.456, 0.406]).to(device)
cnn_normalization_std = torch.tensor([0.229, 0.224, 0.225]).to(device)# create a module to normalize input image so we can easily put it in a
# nn.Sequential
class Normalization(nn.Module):def __init__(self, mean, std):super(Normalization, self).__init__()# .view the mean and std to make them [C x 1 x 1] so that they can# directly work with image Tensor of shape [B x C x H x W].# B is batch size. C is number of channels. H is height and W is width.self.mean = torch.tensor(mean).view(-1, 1, 1)self.std = torch.tensor(std).view(-1, 1, 1)def forward(self, img):# normalize imgreturn (img - self.mean) / self.std# desired depth layers to compute style/content losses :
content_layers_default = ['conv_4']
style_layers_default = ['conv_1', 'conv_2', 'conv_3', 'conv_4', 'conv_5']def get_style_model_and_losses(cnn, normalization_mean, normalization_std,style_img, content_img,content_layers=content_layers_default,style_layers=style_layers_default):# normalization modulenormalization = Normalization(normalization_mean, normalization_std).to(device)# just in order to have an iterable access to or list of content/syle# lossescontent_losses = []style_losses = []# assuming that cnn is a nn.Sequential, so we make a new nn.Sequential# to put in modules that are supposed to be activated sequentiallymodel = nn.Sequential(normalization)i = 0  # increment every time we see a convfor layer in cnn.children():if isinstance(layer, nn.Conv2d):i += 1name = 'conv_{}'.format(i)elif isinstance(layer, nn.ReLU):name = 'relu_{}'.format(i)# The in-place version doesn't play very nicely with the ContentLoss# and StyleLoss we insert below. So we replace with out-of-place# ones here.layer = nn.ReLU(inplace=False)elif isinstance(layer, nn.MaxPool2d):name = 'pool_{}'.format(i)elif isinstance(layer, nn.BatchNorm2d):name = 'bn_{}'.format(i)else:raise RuntimeError('Unrecognized layer: {}'.format(layer.__class__.__name__))model.add_module(name, layer)if name in content_layers:# add content loss:target = model(content_img).detach()content_loss = ContentLoss(target)model.add_module("content_loss_{}".format(i), content_loss)content_losses.append(content_loss)if name in style_layers:# add style loss:target_feature = model(style_img).detach()style_loss = StyleLoss(target_feature)model.add_module("style_loss_{}".format(i), style_loss)style_losses.append(style_loss)# now we trim off the layers after the last content and style lossesfor i in range(len(model) - 1, -1, -1):if isinstance(model[i], ContentLoss) or isinstance(model[i], StyleLoss):breakmodel = model[:(i + 1)]return model, style_losses, content_lossesinput_img = content_img.clone()
# if you want to use white noise instead uncomment the below line:
# input_img = torch.randn(content_img.data.size(), device=device)# add the original input image to the figure:
plt.figure()
imshow(input_img, title='Input Image')def get_input_optimizer(input_img):# this line to show that input is a parameter that requires a gradientoptimizer = optim.LBFGS([input_img])return optimizerdef run_style_transfer(cnn,normalization_mean,normalization_std,content_img,style_img,input_img,num_steps=10000,style_weight=1000000,content_weight=1):"""Run the style transfer."""print('Building the style transfer model..')model, style_losses, content_losses = get_style_model_and_losses(cnn,normalization_mean,normalization_std,style_img,content_img)# We want to optimize the input and not the model parameters so we# update all the requires_grad fields accordinglyinput_img.requires_grad_(True)model.requires_grad_(False)optimizer = get_input_optimizer(input_img)print('Optimizing..')run = [0]while run[0] <= num_steps:def closure():# correct the values of updated input imagewith torch.no_grad():input_img.clamp_(0, 1)optimizer.zero_grad()model(input_img)style_score = 0content_score = 0for sl in style_losses:style_score += sl.lossfor cl in content_losses:content_score += cl.lossstyle_score *= style_weightcontent_score *= content_weightloss = 0.9 * style_score + 0.1 * content_scoreloss.backward()run[0] += 1if run[0] % 50 == 0:print("run {}:".format(run))print('Style Loss : {:4f} Content Loss: {:4f}'.format(style_score.item(), content_score.item()))print()return style_score + content_scoreoptimizer.step(closure)# a last correction...with torch.no_grad():input_img.clamp_(0, 1)return input_imgoutput = run_style_transfer(cnn, cnn_normalization_mean, cnn_normalization_std,content_img, style_img, input_img)plt.figure()
imshow(output, title='Output Image')# sphinx_gallery_thumbnail_number = 4
plt.ioff()
plt.show()

使用pytorch进行风格迁移，以艺术绘画的风格来生成一些好玩的图片相关推荐

风格迁移应用_浅谈风格迁移（一）固定风格迁移
-- 本文来自于VIP Lab的黄宇杰同学撰稿风格迁移是一个有趣的计算机视觉话题.它被用于我们的日常生活中,比如我们常用的美图工具中的各类滤镜背后就是风格迁移技术.其实包括真人到二次元人物.二次元人 ...
keras神经风格迁移_知识分享 | 神经风格迁移-把每一张图片都变成自己喜欢的样子...
原标题:知识分享 | 神经风格迁移-把每一张图片都变成自己喜欢的样子有爱就有阳光灿烂虽然大家总是自嘲,但还是要开开心心的哦~ 选择了打工这条路,也就选择了终身学习 ,今天也让小编带领 ...
零样本风格迁移：多模态CLIP文本驱动图像生成
点击上方"视学算法",选择加"星标"或"置顶" 重磅干货,第一时间送达来源:GitHub ,新智元 [导读]零样本的风格迁移听说过没有?英 ...
风格迁移应用_PyTorch实战图形风格迁移
前言什么是图像风格的迁移?其实现在很多的APP应用中已经普遍存在了,比如让我们选择一张自己的大头照,然后选择一种风格的图片,确认后我们的大头照变成了所选图片类似的风格. 图像风格迁移重点就是找出一张 ...
pythonturtle艺术字_字体风格迁移，卷积神经网络打败艺术字，生成最美汉字
曾几何时,小学的我们上机课时最喜欢摆弄的就是 word 的艺术字,丑陋的效果并不能阻挡我们在每个角落塞进七彩的字体....... "七彩"的艺术字但是时代不同了,我们现在已经有了 ...
图像风格迁移cvpr2020_浅谈风格迁移（二）任意风格迁移
-- 本文来自于VIP Lab的黄宇杰同学撰稿 (接上篇)AdaIN 在17年ICCV中,AdaIN [10]横空出世,完成了任意风格迁移.AdaIN的思路不同于之前的想法,它致力于从一张图片由VGG ...
深度学习项目二：图像的风格迁移和图像的快速风格迁移（含数据和所需源码）
图像风格迁移是指,将一幅内容图的内容,和一幅或多幅风格图的风格融合在一起,从而生成一些有意思的图片一:传统的图像风格迁移为了让我们生成的迁移图在风格上与风格图片尽可能相似,在内容上尽可能与内容图相 ...
汉字风格迁移---dgfont++：用于无监督字体生成的健壮的可变形生成网络
文章目录贡献主要贡献 intro 风格映射笔划.偏旁辅助无监督图像翻译相关工作辅助标注图像翻译变形卷积与注意机制无监督表示学习方法 A概述 B特征变形跳跃连接 C. 样式编码器的 ...
深度学习框架PyTorch入门与实践：第八章 AI艺术家：神经网络风格迁移
本章我们将介绍一个酷炫的深度学习应用--风格迁移(Style Transfer).近年来,由深度学习引领的人工智能技术浪潮越来越广泛地应用到社会各个领域.这其中,手机应用Prisma,尝试为用户的照片 ...

使用pytorch进行风格迁移，以艺术绘画的风格来生成一些好玩的图片

pytorch官网

原理

实现

pytorch的代码：

使用pytorch进行风格迁移，以艺术绘画的风格来生成一些好玩的图片相关推荐

最新文章

热门文章