AI创意助手 (AI creative assistant)

With new and more complex language models coming out in recent times, sometimes it feels like an overwhelming task to make your initial steps into the natural language processing world.

随着最近出现的新的和更复杂的语言模型，有时候进入自然语言处理世界的第一步似乎是一项艰巨的任务。

The purpose of this guide is to provide a fair simple project, a poem generator, to grasp your hands on the NLP world using just one technique: word embeddings.

本指南的目的是提供一个简单的项目(一个诗歌生成器)，以仅使用一种技术(词嵌入)来掌握NLP领域。

那么什么是词嵌入？ (So what are word embeddings?)

Word embeddings are n-dimensional vectorial representations of words, that somehow capture their meaning based on the context existing on a corpus. Stated in another way: words that are used in similar ways in a specific corpus(collection of texts), will have similar vectors.

词嵌入是词的n维矢量表示，它可以基于语料库中存在的上下文以某种方式捕获其含义。 换句话说：在特定语料库(文本集合)中以相似方式使用的单词将具有相似的向量。

And the fun part of having numerical representation of words is that now we can use them in math operations like computing similarity, a measure that we will use in our poem generator.

单词数值表示的有趣之处在于，现在我们可以在数学运算中使用它们，例如计算相似性，这是我们将在诗歌生成器中使用的一种度量。

Similarity is used to quantify the sameness between two vectors. It computes the cosine of the angle between them, which results in a value of 1 for vectors with the same orientation an 0 when the angle is 90°.

相似性用于量化两个向量之间的相似性。 它计算它们之间的角度的余弦值，对于具有相同方向的矢量，当角度为90°时，值为1；值为0。

We will use this property to search for likeness between words and sentences.

我们将使用此属性搜索单词和句子之间的相似性。

算法 (The algorithm)

Our goal is to create a new poem with an input initial word. First, we will randomly select verses, and then we will calculate the similarity between those verses and our initial word. Later, we will select the verse with the highest score and add it to the poem. We will repeat this process but using the last verse as the initial word until the poem is complete.

我们的目标是用输入的首字母写一首新诗。 首先，我们将随机选择经文，然后计算这些经文与初始词之间的相似度。 稍后，我们将选择得分最高的诗歌并将其添加到诗歌中。 我们将重复此过程，但将最后一首诗作为首字母，直到这首诗完成为止。

For this specific guide, I choose to use Edgar Allan Poe poems.

对于本指南，我选择使用埃德加·爱伦·坡的诗歌。

资料收集 (Data recollection)

First, we need to collect the poems that will be the source for our project. To do this we will create a web-scraper in Python, that gather the poems of a specific writer( Edgar Allan Poe in my case) from mypoeticside.com site and save them to a .csv file.

首先，我们需要收集诗歌，这些诗歌将成为我们项目的源头。 为此，我们将使用Python创建一个网络抓取工具，从mypoeticside.com网站上收集特定作家(在我的情况下为Edgar Allan Poe)的诗歌并将其保存到.csv文件中。

数据清理和准备 (Data cleaning and preparation)

We will create a function that takes our previously created .csv file and split the poems into verses, based on a regular expression that we choose. Then, we will also erase and replace some undesired characters from the verses ( colons, for example) in order to make the resulting poem looks more well connected and cohesive.

我们将创建一个函数，该函数将使用我们先前创建的.csv文件，并根据我们选择的正则表达式将诗歌分为诗歌。 然后，我们还将删除并替换诗句中一些不需要的字符(例如冒号)，以使生成的诗看起来更加紧密和凝聚。

We will save the verses in a new .csv file.

我们将这些经文保存在一个新的.csv文件中。

诗歌生成器 (Poem generator)

The poem generator takes the .csv file from the previous step, an initial word and the number of verses of the new poem as inputs.

诗生成器将上一步中的.csv文件，初始单词和新诗的经文数作为输入。

The process starts with our initial word, which ideally should be a widely used word in order to increase the chances of finding a verse with a high similarity. Then, we randomly select n number of verses from the corpus and calculate the similarity between them and our initial word.

该过程从我们的初始单词开始，理想情况下，该单词应该是广泛使用的单词，以增加找到具有高度相似性的经文的机会。 然后，我们从语料库中随机选择n个经文，并计算它们与初始词之间的相似度。

We will use pretained vectors from the Spacy library, specifically the “en_core_web_md” model, that was trained on the Common Crawl dataset. We will also compute the similarity using the same library function.

我们将使用Spacy库中的保留矢量，特别是在Common Crawl数据集上训练的“ en_core_web_md”模型。 我们还将使用相同的库函数来计算相似度。

Finally, we select the highest score verse and repeat this process until the poem is complete.

最后，我们选择得分最高的诗歌并重复此过程，直到诗歌完成为止。

格式化这首诗 (Formatting the poem)

In order to format correctly our newly created poem, we will create a function that uppercase the first verse letter and add a point to the end of the poem.

为了正确格式化我们新创建的诗，我们将创建一个函数，该函数将首诗首字母大写并在诗尾添加一个点。

结果 (Results)

I personally choose to generate short poems with 4 verses, in order to minimize the chances that the result will lack of coherence.

我个人选择用四节诗作短诗，以最大程度地减少结果缺乏连贯性的可能性。

These are some of the results that I liked the most:

这些是我最喜欢的一些结果：

In general, most of the generated poems are entertaining to read, and even though sometimes the result might lack coherence, most of the cases it goes unnoticed how they were created.

总的来说，大多数诗歌都是有趣的阅读作品，尽管有时结果可能缺乏连贯性，但大多数情况下却没有注意到它们是如何创作的。

Hope you enjoyed this guide and feel free to contact me on Linkedin or Twitter.

希望您喜欢本指南，并随时通过Linkedin或Twitter与我联系。