Many tasks in Machine Learning are setup as classification tasks. The name would imply you have to learn a classifier to solve such a task. However, there are many popular alternatives to solving classification tasks which do not involve training a classifier at all!

机器学习中的许多任务都设置为分类任务。 该名称意味着您必须学习分类器才能解决此类任务。 但是，有许多解决分类任务的流行替代方法，这些方法根本不需要训练分类器！

分类任务示例 (An Example Classification Task)

For the purpose of illustrating these alternative methods, let’s use a very simple, yet visual classification problem. Our dataset shall be a random image from the internet. But only a single image… of Ironman!

为了说明这些替代方法，让我们使用一个非常简单但直观的分类问题。 我们的数据集应该是来自互联网的随机图像。 但是只有一个图像……铁人三项！

Each image pixel of this image will count as a unique data sample. Our set of inputs (X) shall be the pixel coordinates and our set of outputs (Y) shall be the pixel colours.

该图像的每个图像像素将被视为唯一的数据样本。 我们的一组输入(X)应该是像素坐标，而我们的一组输出(Y)应该是像素颜色。

The classification task is to predict a pixel colour (y) given a coordinate (x).

分类任务是在给定坐标(x)的情况下预测像素颜色(y)。

• Inputs (X) = pixel coordinates
  输入(X)=像素坐标
• Outputs (Y) = pixel colour class
  输出(Y)=像素颜色类别

We can convert the pixel’s continuous RGBA colour vector into a discrete, categorical class index, so that this toy classification task is simplified.

我们可以将像素的连续RGBA颜色向量转换为离散的分类类别索引，从而简化了此玩具分类任务。

火车测试拆分 (The Train-Test Split)

As a small aside, when splitting the dataset into a training set and a test set, they should have a similar distribution. This means we cant simply split the data down the middle, because the distribution of data that we would test our results on would be very different to the data we used in training (thus leading to bad and/or misleading results)

另外，将数据集分为训练集和测试集时，它们应该具有相似的分布。 这意味着我们不能简单地将数据拆分到中间，因为我们用来测试结果的数据分布将与我们在训练中使用的数据完全不同(从而导致不良和/或误导性的结果)

That is why you should take care when preparing your data (this is all taken care of in the background when using statistical libraries like sklearn)

这就是为什么在准备数据时要小心(在使用sklearn之类的统计库时，所有这些操作都在后台处理)

from sklearn.model_selection import train_test_splitX_train, X_test, Y_train, Y_test = train_test_split(X, Y)

混乱矩阵 (The Confusion Matrix)

We shall use a confusion matrix to plot how well our method classified the colour for each pixel coordinate in the test set. A perfect classification would look like the confusion matrix below:

我们将使用混淆矩阵来绘制我们的方法对测试集中每个像素坐标的颜色分类的程度。 完美的分类看起来像下面的混淆矩阵：

1.直接方法：学习分类器 (1. The Direct Approach: Learn a Classifier)

Now most people solve classification tasks by training a classifier — i.e. Learning a nonlinear function (f) (the classifier) which maps a set of inputs (X) to a set of outputs (Y).

现在，大多数人通过训练分类器来解决分类任务，即学习将一组输入(X)映射到一组输出(Y)的非线性函数(f)(分类器)。

Let us do that too so we have a baseline solution to this classification task to compare against. There are a range of ML models out there to approximate a nonlinear function (the classifier) — but we shall simply use a feed-forward neural network with 3 hidden layers (100 neurons per layer)

让我们也这样做，以便针对此分类任务有一个基准解决方案。 有大量的ML模型可以近似非线性函数(分类器)，但是我们将简单地使用具有3个隐藏层(每层100个神经元)的前馈神经网络。

The results were not bad. They weren’t perfect — but it does a good enough job on our toy classification task (despite occasionally missing off the right eye of ironman)

结果还不错。 它们并不完美-但这在我们的玩具分类任务中做得足够好(尽管偶尔会遗忘Ironman的右眼)

2.替代方法：排名 (2. Alternative Approaches: Ranking)

Ranking (aka Retrieval) is a widely used alternative to training a function (classifier) which directly predicts an output for a given input. By measuring the distance of an input (x) to an output (y), a given input can be ranked against a range of outputs and the best can then be selected (retrieved). Ranking like this is an indirect way to map a given input (x) to an output (y) without requiring a trained classifier (f). Or another way to think about it is — you have decomposed your classifier (f) into an evaluation/distance function (Q) and ranking algorithm (arg max).

排名(又名检索)是训练功能(分类器)的一种广泛使用的替代方法，该功能可直接预测给定输入的输出。 通过测量输入(x)到输出(y)的距离，可以将给定输入与一系列输出进行比较，然后选择最佳(检索)。 这样的排名是一种将给定输入(x)映射到输出(y)的间接方法，而无需训练有素的分类器(f)。 或考虑它的另一种方法是-您已将分类器(f)分解为评估/距离函数(Q)和排名算法(arg max)。

However, doesn’t this mean you just substituted one type of problem for another ? Haven’t you just shifted the problem of learning one function (the classifier) for another function (the distance function)? Yes — but… the latter function may be much easier to learn or you may not even need to learn a distance function if you can use a predefined distance measure (e.g. euclidean distance, cosine similarity, etc).

但是，这是否意味着您只是将一种问题替换为另一种问题？ 您是否刚刚将学习一个功能(分类器)的问题转移到了另一个功能(距离函数)的问题？ 是的，但是…如果您可以使用预定义的距离度量(例如，欧几里德距离，余弦相似度等)，则后者功能可能更容易学习，或者甚至不需要学习距离功能。

2.a K最近的邻居 (2.a K Nearest Neighbours)

K nearest neighbours (KNN) is actually a very well known ranking algorithm used to solve classification tasks. It essentially ranks the new input (x) against a set of K example inputs using a linear distance measure (e.g. euclidean, cosine, etc) and then uses the output (y) of the most similar example input as its prediction.

K最近邻居(KNN)实际上是一种非常著名的排序算法，用于解决分类任务。 它实质上使用线性距离度量(例如，欧几里得，余弦等)针对一组K个示例输入对新输入(x)进行排名，然后将最相似的示例输入的输出(y)用作其预测。

The KNN algorithm is one of the simpler forms of machine learning known as example-based learning (aka retrieval-based learning, instance-based learning, lazy learning, etc), as it relies on memorising examples to avoid learning / approximating a function. It simply uses an off-the-shelf distance function to compare inputs against memorised, exemplar inputs (embedded in the same space).

KNN算法是机器学习的一种较简单形式，称为基于实例的学习(又名基于检索的学习，基于实例的学习，惰性学习等)，因为它依靠记忆示例来避免学习/逼近功能。 它仅使用现成的距离函数将输入与记忆的，示例性的输入(嵌入在同一空间中)进行比较。

from sklearn.neighbors import KNeighborsClassifiernearest_neighbour_algorithm = KNeighborsClassifier(n_neighbors=3)nearest_neighbour_algorithm.fit(X_train, Y_train)Y_predicted = nearest_neighbour_algorithm.predict(X_test)

Siamese Networks actually work on a very similar principal to KNNs (i.e. they compare new inputs to exemplar inputs to retrieve their corresponding outputs), however it learns its own, custom distance function.

暹罗网络实际上在与KNN相似的原理上工作(即，它们将新输入与示例输入进行比较以检索其相应的输出)，但是它学习了自己的自定义距离函数。

2.b联合嵌入 (2.b Joint Embeddings)

We can get away with using custom distance functions by comparing inputs (x) with other inputs (x) which are embedded in the same space. However, this is more tricky when measuring the distance between inputs (x) with outputs (y) directly due to them being embedded in two different spaces. We can overcome this by using joint embeddings (i.e. embedding the inputs and outputs in a shared space).

通过将输入(x)与嵌入在同一空间中的其他输入(x)进行比较，我们可以使用自定义距离函数。 但是，由于直接将输入(x)嵌入两个不同的空间中，因此在直接测量输入(x)与输出(y)之间的距离时，这比较棘手。 我们可以通过使用联合嵌入(即将输入和输出嵌入到共享空间中)来克服这一问题。

Therefore this method involves learning joint embeddings such that the inputs (X) and outputs (Y) can be embedded into a shared space. Then we can rank a new input against the outputs using linear distance metrics (e.g. euclidean distance, cosine distance, etc). There are a few different ways to embed the outputs and inputs onto a shared manifold — such as taking the learnt feature representations of a neural model (e.g. Recurrent Embedding Dialogue Policy, etc).

因此，该方法涉及学习联合嵌入，以便可以将输入(X)和输出(Y)嵌入到共享空间中。 然后，我们可以使用线性距离度量(例如，欧几里得距离，余弦距离等)对输出进行新排序。 有几种不同的方法可以将输出和输入嵌入到共享的流形中，例如采用神经模型的学习到的特征表示(例如，循环嵌入对话策略等)。

We shall be using Matrix Factorisation.

我们将使用矩阵分解。

2.c学习距离函数 (2.c Learning a Distance Function)

You can also approximate a custom, nonlinear distance measure which would have learnt to measure the relative distances of input-output pairs (despite being embedded differently). We can do this by feeding in a set of concatenated input-output vectors as the model input, and training the model to predict a distance score (a value between 0 and 1) for the concatenated input-output pair.

您也可以近似自定义的非线性距离度量，该度量将学会测量输入-输出对的相对距离(尽管嵌入方式有所不同)。 我们可以通过输入一组串联的输入-输出向量作为模型输入，并训练模型以预测串联的输入-输出对的距离得分(0到1之间的值)来做到这一点。

We therefore modify the training data by concatenating inputs and outputs randomly (to become our new training inputs) and setting their expected distance to 0 if the input-output pair is a valid mapping,and 1 if its an invalid input-output mapping. The model used to approximate the distance function is simply a feedforward neural network (with 3 hidden layers, each with 100 neurons) almost identical to the one used for the classifier method (except this neural network is set up for learning a regression task as opposed to a classification task).

因此，我们通过以下方式修改训练数据：将输入和输出随机串联(以成为新的训练输入)，如果输入/输出对是有效映射，则将其期望距离设置为0；如果输入/输出对是无效输入，则将其期望距离设置为1。 用于近似距离函数的模型只是一个前馈神经网络(具有3个隐藏层，每个具有100个神经元)，几乎与用于分类器方法的模型相同(除了该神经网络是为学习回归任务而设置的)到分类任务)。

结论 (Conclusion)

Performance wise, the results of the alternative methods were similar to the classifier, however, retrieval based methods do present additional benefits over a classifier in other areas, like adding a stochastic element (if the top ranking result is always chosen it is identical to using a classifier or another deterministic mapping, however, adding some uncertainty can be done by ensuring the top ranking result is not always chosen), or like updating classes more easily, solving much larger multiclass classification problems (i.e. those with hundreds of classes) and being able to solve multilabel classification problems (i.e. predicting the top N applicable outputs for a given input), etc.

在性能方面，替代方法的结果类似于分类器，但是基于检索的方法在其他领域确实比分类器具有更多优势，例如添加随机元素(如果始终选择排名最高的结果，则等同于使用分类器或其他确定性映射，但是，可以通过确保不总是选择排名最高的结果来增加一些不确定性，或者像更轻松地更新类，解决更大的多类分类问题(即具有数百个类的问题)并能够解决多标签分类问题(即预测给定输入的前N个适用输出)等。

谢谢 (Thanks)

Thanks for reading. All the code is included in the attached Colab notebook. Have fun experimenting

谢谢阅读。 所有代码都包含在随附的Colab笔记本中。 尝试愉快