生成对抗网络生成多维数据集

介绍(Introduction)

GAN architecture has been the standard for generating content through AI, but can it actually invent new content outside what’s available in the training dataset? Or it’s just imitating the training data and mixing the features in new ways?

GAN架构一直是通过AI生成内容的标准，但它实际上可以在训练数据集中可用的范围之外发明新内容吗？还是只是模仿训练数据并以新方式混合功能？

In this article, I will discuss “Rewriting Deep Generative Model” paper that enables editing GAN model directly to give the output that we want even if it doesn’t match the existing dataset. The image above is an example of an edit where you copy the helmet feature and pasted it on the horse contextually. I believe this possibility will open many new interesting applications in the digital industry such as generating fictitious content for animations or games where there may be no existing dataset. You can watch the presentation video at the link below.

在本文中，我将讨论“重写深度生成模型”一文，该文件可直接编辑GAN模型，以提供所需的输出，即使它与现有数据集不匹配也是如此。上图是编辑示例，您可以在其中复制头盔功能并将其粘贴到上下文中。我相信这种可能性将打开数字行业中许多新的有趣应用程序，例如为可能不存在现有数据集的动画或游戏生成虚拟内容。您可以在下面的链接上观看演示视频。

甘 (GAN)

Generative Adversarial Network (GAN) is a generative model which means it can generate a similar realistic output as the training data. For example, a GAN trained on human faces would be able to generate similar looking realistic faces. GAN is able to do this by learning the distribution of the training data and generate new content that follows the same distribution.

生成对抗网络(GAN)是一种生成模型，这意味着它可以生成与训练数据类似的现实输出。例如，经过人脸训练的GAN将能够生成相似外观的逼真的面Kong。 GAN可以通过学习训练数据的分布并生成遵循相同分布的新内容来做到这一点。

GAN “indirectly” learns the distribution by having a discriminator that tries to differentiate real and fake images and a generator that creates fake data to fool the discriminator. These 2 networks will continuously compete and learn from each other until both of them can generate and discriminate realistic images respectively.

GAN通过拥有一个试图区分真实图像和伪造图像的鉴别器以及一个创建伪造数据来欺骗鉴别器的生成器来“间接”学习分布。这两个网络将不断竞争和相互学习，直到它们两个都可以分别生成和区分现实图像为止。

GAN的局限性 (GAN Limitations)

Although GAN is able to learn the general data distribution and generate diverse images of the dataset. It is still limited to what exists in the training data. For example, let’s take a GAN model trained on faces. Although it can generate new faces that do not exist in the dataset, it cannot invent an entirely new face with novel features. You can only expect it to combine what the model already know in new ways.

尽管GAN能够学习一般数据分布并生成数据集的各种图像。它仍然限于训练数据中存在的内容。例如，让我们采用经过训练的GAN模型。尽管它可以生成数据集中不存在的新面Kong，但它不能发明具有新颖特征的全新面Kong。您只能期望它以新的方式结合模型已经知道的内容。

Hence, there is no problem if we only want to generate normal faces. But what if we want faces with bushy eyebrows or maybe a third eye? The GAN model cannot generate this as there is no sample with bushy eyebrows or a third eye in the training data. The quick solution would be simply editing the generated face with a photo-editing tool, but it’s not feasible if we want to generate tons of images like it. Hence, GAN model would suit the problem better, but how do we make GAN generates our desired images when there is no existing dataset?

因此，如果我们只想生成法线脸就没有问题。但是，如果我们想要眉毛浓密或第三只眼的脸怎么办？ GAN模型无法生成此模型，因为在训练数据中没有带有浓密眉毛或第三只眼睛的样本。快速的解决方案是使用照片编辑工具简单地编辑生成的面部，但是如果我们要生成大量像这样的图像，这是不可行的。因此，GAN模型将更适合该问题，但是当没有现有数据集时，我们如何使GAN生成所需的图像？

重写GAN规则 (Rewriting GAN Rules)

In January 2020, MIT and Adobe Research published an interesting paper titled “Rewriting Deep Generative Model” which enable us to edit GAN model directly and generate novel content. What does it mean by model rewriting? Instead of letting the model optimize itself based on the training data or label, we directly set the rules (parameters) that we want to keep to give us the desired results. Want a helmet on a horse? No problem. We can copy the features for the helmet and put it on the horse head feature. However, this requires an understanding of the internal parameters and how it affects the output, which has been quite a challenge in the past. Although, the paper has proven that it is feasible.

2020年1月，麻省理工学院和Adobe Research发表了一篇有趣的论文，名为“重写深度生成模型” ，使我们能够直接编辑GAN模型并产生新颖的内容。模型重写是什么意思？与其让模型根据训练数据或标签进行优化，不如直接设置我们要保留的规则(参数)以提供期望的结果。想要戴上头盔吗？没问题。我们可以复制头盔的特征并将其放在马头特征上。但是，这需要了解内部参数及其对输出的影响，这在过去是一个很大的挑战。虽然，本文已证明这是可行的。

Copy and paste features contextually by rewriting the model [Image by Author]

The difference between training and rewriting is akin to the difference between natural selection and genetic engineering. While training allows efficient optimization of a global objective, it does not allow direct specification of internal mechanisms. In contrast, rewriting allows a person to directly choose the internal rules they wish to include, even if these choices do not happen to match an existing data set or optimize a global objective.

培训和重写之间的区别类似于自然选择和基因工程之间的区别。尽管培训允许有效地优化全局目标，但不允许直接指定内部机制。相比之下，重写使一个人可以直接选择他们希望包括的内部规则，即使这些选择并非恰好匹配现有数据集或优化全局目标。

- David Bau (Lead author of the paper)

-David Bau(论文的主要作者)

As David Bau said, rewriting a model is like genetic engineering. It’s like inserting DNA of the glowing jellyfish into a cat to make a cat that glows in the dark.

正如David Bau所说，重写模型就像基因工程。这就像将发光的水母的DNA插入猫中，使猫在黑暗中发光。

*Anahid PahhlawanianAnahid Pahhlawanian提供*]]

怎么运行的 (How It Works)

How do you actually rewrite a generative model? The paper proposes the idea of treating the weights of the generator as Optimal Linear Associative Memory (OLAM). The role of OLAM is to store a key-value pairs association. We will select a certain layer L, which represents the value V that denotes the output features of the image such as smile expression. Then, the previous layer before layer L will represent the key K, which denotes a meaningful context such as the mouth location. Here, the weight W, between layer L and layer L-1 acts as a linear associative memory that stores the association between K and V.

您实际上如何重写生成模型？本文提出了将生成器的权重视为最佳线性联想记忆(OLAM)的想法。 OLAM的作用是存储键值对关联。我们将选择一个特定的图层L，该图层代表值V ，该值V表示图像的输出特征，例如微笑表情。然后，在层L之前的前一层将表示密钥K ，密钥K表示有意义的上下文，例如嘴巴位置。此处，L层和L-1层之间的权重W用作存储K和V之间的关联的线性关联存储器。

Weight as associative memory [Image by Author]

We can think K

生成对抗网络生成多维数据集_生成没有数据集的新颖内容相关推荐

生成对抗网络的背景与意义_生成对抗网络的意义
生成对抗网络的背景与意义为什么我们需要生成对抗网络:-(Why do we need Generative Adversarial Network: -) If we show a lot and ...
生成对抗网络的损失函数如何设计_生成对抗网络（GAN）
GAN的全称是 Generative Adversarial Networks,中文名称是生成对抗网络.原始的GAN是一种无监督学习方法,巧妙的利用"博弈"的思想来学习生成式模型. ...
生成对抗网络的损失函数如何设计_理解生成对抗网络，一步一步推理得到GANs（一）...
作者:Joseph Rocca 编译:ronghuaiyang 导读 GANs在2014年提出,然后就在图像生成领域取得了巨大的成功,但是其背后的原理却不是那么好理解,这篇文章带你从最基础的随机变量开 ...
生成对抗网络的损失函数如何设计_如何检测极小人脸？试试超分辨率
原创声明:本文为 SIGAI 原创文章,仅供个人学习使用,未经允许,不能用于商业目的. 其它机器学习.深度学习算法的全面系统讲解可以阅读<机器学习-原理.算法与应用>,清华大学出版社,雷明 ...
GAN（生成对抗网络） and CGAN（条件生成对抗网络）
前言 GAN(生成对抗网络)是2014年由Goodfellow大佬提出的一种深度生成模型,适用于无监督学习.监督学习.但是GAN进行生成时是不可控的,所以后来又有人提出可控的CGAN(条件生成对抗网络 ...
调研-笔记-基于生成对抗网络的恶意域名训练数据生成
DGA 域名字符生成模型域名字符分析问题:理论上 GAN 中的生成器和判别器部分采用任意可微函数都能表示,因此其主要用于连续数据的处理,如图像生成.视频检测等[13].基于文本的离散数据处理一直是 ...
生成对抗网络(GAN)——MNIST手写数字生成
前言正文一.什么是GAN 二.GAN的应用三.GAN的网络模型对抗生成手写数字一.引入必要的库一.引入必要的库二.进行准备工作三.定义生成器和判别器模型四.设置损失函数和优化器,以及 ...
wps生成正态分布的随机数_量子计算与机器学习: 量子生成对抗网络QGAN
随着量子信息和量子计算的快速发展(经费多了),科研工作者们一边感叹着量子计算机时代即将拥有的强大计算能力,一边又在考虑着如何将现有的高效算法和量子计算机相适配.作为最近几年如此火爆的机器学习,也就自然 ...
四天搞懂生成对抗网络（一）——通俗理解经典GAN
点击左上方蓝字关注我们 [飞桨开发者说]吕坤,唐山广播电视台,算法工程师,喜欢研究GAN等深度学习技术在媒体.教育上的应用. 序言做图像分类.检测任务时,为了提高模型精度,在数据处理方面,我尝试了很 ...
新手必看：生成对抗网络的初学者入门指导
新手必看:生成对抗网络的初学者入门指导 https://www.cnblogs.com/DicksonJYL/p/9698877.html 本文为 AI 研习社编译的技术博客,原标题 A Beginn ...

生成对抗网络生成多维数据集_生成没有数据集的新颖内容