数据科学 (Data Science)

Word embeddings.

词嵌入。

What are they? What do they look like? How are they made?…

这些是什么？ 他们看起来怎么样？ 它们是怎么制成的？…

I’ll break it down without code or lingo.

我将在没有代码或行话的情况下将其分解。

This is based on my experience explaining machine learning terms to non-technical colleagues.

这是基于我向非技术同事解释机器学习术语的经验。

了解：机器学习模型仅了解数字 (Understand: Machine learning models only understand numbers)

You can’t feed text to an ML model.

您无法将文本输入ML模型。

Because, in a nutshell, math doesn’t work on text.

简而言之，因为数学不适用于文本。

You must convert words/sentences/etc… to a list of numbers first.

您必须先将单词/句子/等转换为数字列表。

Word embeddings are one approach to converting text to a machine-readable format.

单词嵌入是将文本转换为机器可读格式的一种方法。

There are several approaches, including bags of words, but we won’t get into those here.

有几种方法，包括大量的单词 ，但是我们在这里不做介绍。

单词嵌入将单词转换为数字序列 (A word embedding converts words to sequences of numbers)

Below are a few words, “bright”, “shiny” and “dog”, translated into numbers with popular word-embedding library, Word2Vec.

以下是一些流行的词嵌入库Word2Vec将“亮”，“发光”和“狗”这几个词翻译成数字。

When using pre-trained embeddings like Word2Vec, the numeric representation of each word comes included.

当使用像Word2Vec这样的经过预训练的嵌入时，每个单词的数字表示都包括在内。

You simply convert your words into each given sequence of numbers, then plug them into your model.

您只需将单词转换为每个给定的数字序列，然后将其插入模型即可。

词嵌入中的值是什么意思？ (What do the values in a word embedding mean?)

In isolation, nothing!

孤立无事！

But relative to each other, they contain a lot of information.

但是相对而言，它们包含很多信息。

In the previous Word2Vec example, we can see that at each index, “bright” and “shiny” are more similar to each other than to “dog”.

在前面的Word2Vec示例中，我们可以看到，在每个索引处，“明亮”和“发光”彼此之间的相似程度要比与“狗”相似。

If we plotted D1 and D2 on a 2x2 grid, we would see “bright” and “shiny” very close to each other, and “dog” further away.

如果在2x2的网格上绘制D1和D2，我们将看到“明亮”和“发亮”彼此非常接近，而“狗”则更远。

In the word-embedding approach, words with a similar meaning “should” have similar numeric values.

在单词嵌入方法中，具有相似含义“应该”的单词具有相似的数值。

为什么相似的单词具有相似的数值？ (Why do similar words have similar numeric values?)

Good question.

好问题。

Because a word IS its context.

因为一个词就是它的上下文。

“context” = “the words used around a word” in the training text used to build the word-embedding.

在用于构建词嵌入的训练文本中，“ context” =“围绕单词使用的单词”。

So when deriving numeric values for “beautiful” and “gorgeous” we would likely find similar words around them, and they would get similar numeric values.

因此，当导出“美丽”和“华丽”的数值时，我们可能会在它们周围找到相似的词，并且它们将获得相似的数值。

A word’s numeric values are a function of the weights in a neural network upon correctly guessing a word, given its context (CBOW approach).

给定上下文(CBOW方法)，在正确猜出单词后，单词的数值是神经网络中权重的函数。

The neural network keeps adjusting its weights until it can take the context of a word, and guess the word itself.

神经网络会不断调整权重，直到可以获取单词的上下文并猜测单词本身为止。

At that point, the current weights (think knobs and dials on a machine) become the numeric values.

此时，当前的砝码(机器上的旋钮和刻度盘)将变为数值。

Similar context is often found around words with similar meaning.

在具有相似含义的单词周围通常会发现相似的上下文。

So similar neural network weights will predict similar words.

因此，相似的神经网络权重将预测相似的单词。

结论 (Conclusion)

Disclaimer: I’ve made some generalizations and skipped some small steps to help make this more explainability

免责声明：我做了一些概括，并跳过了一些小步骤来帮助提高解释性

That’s it.

而已。

Word embeddings are an approach (one of several) for converting text into numbers so computers can process it.

单词嵌入是一种将文本转换为数字以便计算机可以处理的方法(几种)。

Practical experience (successes and failures) is your best guide as to when you should actually use them, versus other approaches. While very popular, they often underperform more traditional approaches (in my anecdotal experience).

与何时使用其他方法相比，实践经验(成功和失败)是您何时应该真正使用它们的最佳指南。 尽管很受欢迎，但它们通常不如传统方法(根据我的轶事)。

That’s why I always recommend diving in and getting your hands dirty on some code.

这就是为什么我总是建议您深入了解一些代码的原因 。

Remember that in machine learning, there is no free lunch. We really don’t know what works until we try it!

请记住，在机器学习中， 没有免费的午餐 。 在尝试之前，我们真的不知道什么有效！