Natural language processing (NLP) is one of the biggest areas of machine learning research, and although current linguistic machine learning models achieve numerically-high performance on many language-understanding tasks, they often lack optimization for reducing implicit biases.

自然语言处理(NLP)是机器学习研究的最大领域之一，尽管当前的语言机器学习模型在许多理解语言的任务上实现了数值上的高性能，但它们通常缺乏优化以减少隐性偏差。

Let’s start from the beginning.

让我们从头开始。

What is bias in machine learning models? Essentially, it’s when machine learning algorithms express implicit biases that often pass undetected during testing because most papers test their models for raw accuracy. Take, for example, the following instances of deep learning models expressing gender bias. According to our deep learning models,

机器学习模型中的偏见是什么？ 从本质上讲，这是机器学习算法表达隐性偏差的时候，该偏差通常在测试过程中未被发现，因为大多数论文都在测试其模型的原始准确性。 以下列表示性别偏见的深度学习模型为例。 根据我们的深度学习模型，

• “He is doctor” has a higher likelihood than “She is doctor.” [Source]

  “他是医生”比“她是医生”的可能性更高。 [ 来源 ]

• Man is to woman as computer programmer is to homemaker. [Source]

  男人是女人，计算机程序员是家庭主妇。 [ 来源 ]

• Sentences with female nouns are more indicative of anger. [Source]

  带有女性名词的句子更能表示愤怒。 [ 来源 ]

• Translating “He is a nurse. She is a doctor” into Hungarian and back to English results in “She is a nurse. He is a doctor.” [Source]

  翻译“他是一名护士。 她是匈牙利的一名医生”，而英语为“她是一名护士”。 他是一个医生。” [ 来源 ]

In these examples, the algorithm is essentially expressing stereotypes, which differs from an example such as “man is to woman as king is to queen” because king and queen have a literal gender definition. Kings are defined to be male and queens are defined to be female. Computer programmers are not defined to be male and homemakers are not defined to be female, so “Man is to woman as computer programmer is to homemaker” is biased.

在这些示例中，算法本质上是在表达刻板印象，这与“男人是女人，国王是王后”这样的示例有所不同，因为国王和王后都有字面上的性别定义。 国王被定义为男性，女王被定义为女性。 没有将计算机程序员定义为男性，而将家庭佣工未定义为女性，因此，“男人对女人就像对计算机编程员对家庭主妇一样”是有偏见的。

Other forms of bias other than gender bias are also prominent in our models. Here are examples of some other forms of bias:

在我们的模型中，除性别偏见外的其他偏见也很突出。 以下是一些其他形式的偏见的示例：

• According to machine learning models, black is to criminal as Caucasian is to police. [Source]

  根据机器学习模型，黑人是罪犯，白种人是警察。 [ 来源 ]

• According to machine learning models, lawful is to Christianity as terrorist is to Islamic. [Source]

  根据机器学习模型，合法对基督教来说就像恐怖分子对伊斯兰教一样。 [ 来源 ]

• Tweets written by African Americans are more likely to be flagged as offensive by AI. [Source]

  非裔美国人撰写的推文更有可能被AI标记为令人反感。 [ 来源 ]

Now if you’re anything like me, you’re probably thinking: But how can machines be biased if they don’t have emotions?

现在，如果您像我一样，您可能会在想： 但是，如果机器没有情感，该如何偏见呢？

The key is that machine learning models learn patterns in the data. So let’s say our data tends to put female pronouns around the word “nurse” and male pronouns around the word “doctor.” Our model will learn those patterns from and learn that nurse is usually female and doctor is usually male. Whoops. By no fault of our own, we’ve accidentally trained our model to think doctors are male and nurses are female.

关键在于机器学习模型可以学习数据中的模式 。 因此，可以说我们的数据倾向于在“护士”一词旁加上女性代词，在“医生”一词旁加上男性代词。 我们的模型将从中学习这些模式，并了解护士通常是女性，医生通常是男性。 哎呀 。 毫无疑问，我们无意中训练了我们的模型，以为医生是男性，护士是女性。

So how do you address this? Like many problems, bias in NLP can be addressed at the early stage or at the late stages. In this instance, the early stage would be debiasing the dataset, and the late stage would be debiasing the model.

那么您如何解决这个问题？ 像许多问题一样，可以在早期或后期解决NLP中的偏见。 在这种情况下，早期将对数据集进行反偏，而晚期将对模型进行反偏。

Solution A: debias the datasets. For this to be successful, we first have to remove existing datasets that contain biases. For example, MIT recently withdrew a popular computer vision dataset called Tiny Images after learning that it was filled with social biases, including racist, misogynistic, and demeaning labels. This doesn’t mean we can’t use those datasets, but it means that we should remove them and edit them to account for bias. Similarly, new datasets must be checked to account for bias. As of now, the most agreed-upon way to debias datasets is diversifying the dataset. For example, if a dataset consistently puts female pronouns around the word “nurse,” it can be debiased by adding data in which nurses are male.

解决方案A：对数据集进行除偏。 为使此成功，我们首先必须删除包含偏差的现有数据集。 例如，麻省理工学院(MIT)在得知流行的计算机视觉数据集充斥着社会偏见之后，就撤回了该数据集，其中包括种族主义，厌女症和贬低性标签。 这并不意味着我们不能使用这些数据集，而是意味着我们应该删除它们并进行编辑以解决偏差。 同样，必须检查新的数据集以解决偏差。 到目前为止，最公认的去偏斜数据集的方法是使数据集多样化。 例如，如果数据集始终在“护士”一词周围放置女性代词，则可以通过添加护士是男性的数据来​​消除偏见。

Solution B: debias the models. This is done by modifying the actual vector-representations of words. For example, the Hard Debias algorithm and Double-Hard Debias algorithm modify the vector-representations to remove stereotyping-information (such as the link between “receptionist” and “female”) while maintaining useful gender information (such as the link between “queen” and “female”). These algorithms show promising results and are definitely a good step in addressing NLP bias.

解决方案B：对模型进行偏移。 这是通过修改单词的实际矢量表示来完成的。 例如， Hard Debias算法和Double-Hard Debias算法修改矢量表示以删除定型信息(例如“接待员”和“女性”之间的链接)，同时保持有用的性别信息(例如“ queen”之间的链接) ”和“女性”)。 这些算法显示出令人鼓舞的结果，并且绝对是解决NLP偏差的好一步。

Do we still have time to address bias? Though NLP has progressed rapidly as a field, it’s never too late to address bias in NLP models. No matter how we address these bias issues, however, we still have to address them as early as possible, preferably before the models reach a real-world setting. Here’s an example where bias wasn’t caught, and a biased model ended up reaching real-world application and having big consequences:

我们还有时间解决偏见吗？ 尽管NLP作为一个领域发展Swift，但解决NLP模型中的偏差永远不会太晚。 但是，无论我们如何解决这些偏差问题，我们仍然必须尽早解决它们，最好是在模型达到实际设置之前。 这是一个示例，其中的偏见没有被发现，有偏见的模型最终进入了现实世界的应用程序并产生了严重的后果：

COMPAS, an artificial intelligence system used in various states, is designed to predict whether or not a perpetrator is likely to commit another crime. The system, however, turned out to have an implicit bias against African Americans, predicting double the amount of false positives for African Americans than for Caucasians. Because this implicit bias was not caught before the system was deployed, many African Americans were unfairly and incorrectly predicted to re-offend.

COMPAS是一种在各个州使用的人工智能系统，旨在预测犯罪者是否有可能再次犯罪。 然而，事实证明，该系统对非裔美国人有隐性偏见，预测非裔美国人的假阳性数量是高加索人的两倍。 由于在部署系统之前并未发现这种隐性偏见，因此许多非裔美国人被不公平且错误地预测会再次冒犯。

COMPAS, an AI to help law enforcement identify low-risk and high-risk criminals, turned out to be implicitly biased against African Americans. Image obtained from ProPublica, the organization that discovered these biases.

COMPAS是帮助执法机构识别低风险和高风险罪犯的AI，事实证明它隐含地偏向非裔美国人。 从 发现这些偏见的组织 ProPublica 获得的图像 。

Bias in NLP is a pressing issue that must be addressed as soon as possible. The consequences of letting biased models enter real-world settings are steep, and the good news is that research on ways to address NLP bias is increasing rapidly. Hopefully, with enough effort, we can ensure that deep learning models can avoid the trap of implicit biases and make sure that machines are able to make fair decisions.

NLP中的偏差是一个紧迫的问题，必须尽快解决。 让有偏见的模型进入现实世界的后果非常严峻，而且好消息是，解决NLP偏见的方法的研究正在Swift增加。 希望我们能够通过足够的努力来确保深度学习模型能够避免隐式偏差的陷阱，并确保机器能够做出公平的决策。

Further reading:

进一步阅读：

• This group of researchers has a bunch of papers on bias in NLP.

  这组研究人员有大量关于NLP偏见的论文。

• The full article by ProPublica on the biases in the COMPAS system.

  ProPublica撰写的有关COMPAS系统偏差的全文。