Many businesses and organizations are turning to machine learning for solutions to challenging business goals and problems. Providing machine learning solutions to meet these needs requires that one follows a systematic process from problem to solution. The stages of a machine learning project constitute the machine learning pipeline. The machine learning pipeline is a systematic progression of a machine learning task from data to intelligence.

许多企业和组织正在转向机器学习来寻求具有挑战性的业务目标和问题的解决方案。 提供满足这些需求的机器学习解决方案要求从问题到解决方案遵循一个系统的过程。 机器学习项目的各个阶段构成了机器学习管道。 机器学习管道是机器学习任务从数据到智能的系统性演进。

During our training as ML engineers, a lot of focus is invested in learning about algorithms, techniques, and machine learning tools but often, less attention is given to how to approach industry and business problems from the problem to a usable solution.

在我们作为ML工程师的培训期间，我们投入了大量的精力来学习算法，技术和机器学习工具，但通常很少关注如何从问题到可用的解决方案来解决行业和业务问题。

In this article, I present the machine learning pipeline that provisions for a comprehensive approach to solving real-world problems using machine learning. I will start with the observable or explainable problem as companies/businesses are likely to present them to an engineer and will walk you through the stages that a project needs to go through up till it ends as a usable solution available to platform end-users.

在本文中，我介绍了机器学习管道，该管道提供了一种全面的方法来使用机器学习解决实际问题。 我将从一个可观察或可以解释的问题开始，因为公司/企业很可能将它们呈现给工程师，并将引导您完成项目需要经历的各个阶段，直到它作为平台最终用户可用的可用解决方案结束为止。

You will basically see at a top-level what stages were involved in building, for instance, the Netflix movie recommendation engine that runs in the background of the movie platform and personalizes your experience, showing you the movies you are likely to be interested in.

您基本上会在高层看到构建的各个阶段，例如，在电影平台的后台运行的Netflix电影推荐引擎，它将个性化您的体验，向您展示您可能感兴趣的电影。

Solving any business problem follows these fundamental stages and so it is necessary for all practitioners to understand and leverage it. If you sharpen your thinking about machine learning projects in light of this article, I believe that you will be more effective, structured when doing ML projects. You will understand from this article how to relate more with industrial stakeholders who may not understand the whole ML buzz but are genuinely seeking relevant and desired solutions to good problems.

解决任何业务问题都遵循这些基本阶段，因此所有从业人员都必须理解并利用它。 如果您根据本文加强对机器学习项目的思考，我相信您在进行ML项目时会更有效率，更有条理。 您将从本文中了解如何与行业利益相关者建立更多联系，他们可能不了解整个ML嗡嗡声，但实际上正在寻求相关的问题和理想的解决方案。

I know from my experience when I did my first internship as a data analyst for one of Africa’s leading data center colocation service providers, Africa Data Centers, the frustration inherent in not following this paradigm. I did not think then that my approach was not optimal, because I did not know it then. I can only imagine how much time and frustration it would have saved me and how considerably improved my performance and output would have been if I had this understanding then.

我从作为非洲领先的数据中心托管服务提供商之一的非洲数据中心的数据分析师的第一份实习经历时就知道，由于不遵循这种范例而产生的挫败感。 当时我并不认为我的方法不是最优的，因为那时我还不知道。 我只能想象如果我有了这种理解，它将节省我多少时间和挫败感，以及可以大大改善我的性能和输出。

The stages of a machine learning project are summarized by the figure below.

下图总结了机器学习项目的各个阶段。

业务问题或研究问题 (The Business problem or research problem)

Start with the business problem__

从业务问题开始__

In many cases, organizations tend to present this as a goal, what they want to achieve. Very often there is a story to it and that story is important. This is how we have been doing things or how the system was behaving, and this is what we would like to achieve. In my case, it was something like;

在许多情况下，组织倾向于将其作为目标，即他们想要实现的目标。 很多时候都有一个故事，这个故事很重要。 这就是我们的工作方式或系统的行为方式，这就是我们想要实现的目标。 就我而言，这有点像；

“We will like to use the historical data we have on our energy consumption to determine our options for energy efficiency optimization and cost-saving.”

“我们希望利用我们在能源消耗方面的历史数据来确定我们在能源效率优化和成本节省方面的选择。”

Simply put, the business problem was what can we do to reduce expenses on energy?

简而言之，业务问题是我们该如何减少能源支出？

构架机器学习问题 (Framing the machine learning problem)

From the business problem, you frame the machine learning problem. This is where domain knowledge/expertise comes in. This is not trivial at all because to get the right solution you must start with the right problem/questions.

从业务问题中，您可以构架机器学习问题。 这就是领域知识/专长的来源。这根本不是一件容易的事，因为要获得正确的解决方案，您必须从正确的问题/问题开始。

“Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question”__J. Tukey, The Future of Statistical Analysis

“对于正确的问题 (通常是模糊的)，比对错误的问题的精确答案要好得多”。 Tukey，统计分析的未来

Many organizations that are using machine learning seriously sometimes consult domain experts to help them ask the right questions. However, it may not be the case that when faced with a problem, you would bring in an expert. You may be the expert/consultant that was brought to figure things out. In that case, research is the only way to go. What is the industry doing to solve the same or similar problems?

许多认真使用机器学习的组织有时会咨询领域专家，以帮助他们提出正确的问题。 但是，遇到问题时，不一定会聘请专家。 您可能是被带去解决问题的专家/顾问。 在这种情况下， 研究是唯一的方法 。 行业如何解决相同或相似的问题？

Your goal is to not waste time answering the wrong question, right? Translating a business problem to a machine learning problem is so important that it determines the fate of your entire project. Completing this step should start leading you towards the kind of data that will be necessary to answer the machine learning question. From all your research and understanding, you should already have relevant features to expect in your dataset.

您的目标是不要浪费时间回答错误的问题，对吗？ 将业务问题转换为机器学习问题非常重要，以至于它决定了整个项目的命运。 完成此步骤应开始引导您获得回答机器学习问题所需的数据类型。 从您的所有研究和理解中，您应该已经具有期望在数据集中使用的相关功能。

数据收集和/或整合 (Data collection and/or integration)

• Is the data relevant to the problem?

  数据与问题有关吗？

• Is the data enough to train a good model?

  数据足以训练一个好的模型吗？

This step involves putting together already existing data or collecting the necessary data. If there is already existing data, you will have to determine if the data is relevant to the machine learning problem and thus the business problem. This is very important especially if the organization is not a typical machine learning organization to have predetermined that before collecting the data they now have. It is not uncommon (been there once) to find that an organization has collected data that is not relevant to the problem they want to solve.

此步骤涉及将已经存在的数据放在一起或收集必要的数据。 如果已经存在数据，则必须确定数据是否与机器学习问题以及业务问题相关。 这是非常重要的，特别是如果该组织不是典型的机器学习组织，那么在收集数据之前要预先确定它们。 发现组织收集的数据与他们要解决的问题无关的情况并不少见(一次见过)。

A good rule of thumb is to ask the question “What data will a human expert need to solve the problem if this task was left to them?”. If a human expert cannot use the data available to deduce correct predictions, it is almost definite that a machine cannot. Again, an expert will provide you with better information about how they will solve the problem and what data they will need to answer the question you are trying to answer using machine learning.

一个好的经验法则是问一个问题：“如果这项任务留给他们，专家将需要什么数据来解决问题？”。 如果人类专家无法使用可用数据来得出正确的预测，则几乎可以肯定机器无法做到。 同样，专家将为您提供更好的信息，说明他们如何解决问题以及使用机器学习来回答您要回答的问题所需的数据。

The quality of the model or analysis performed is totally dependent on the quality of the data. Just like one cannot make fine wine with low-quality grapes, one cannot build a good model with poor quality data.

模型或执行的分析的质量完全取决于数据的质量。 就像一个人不能用低质量的葡萄来酿造优质葡萄酒一样，一个人也不能用低质量的数据来建立一个好的模型 。

It might be possible to deduce more valuable features from the original data using feature engineering. Therefore, also think critically to see if relevant features are simply hidden in the dataset. Nevertheless, it is better to advise what data the organization/business should collect that will help their quest better.

使用特征工程可以从原始数据中推断出更多有价值的特征。 因此，还必须进行批判性思考，以查看相关特征是否仅隐藏在数据集中。 但是，最好建议组织/企业应收集哪些数据，以更好地帮助他们进行搜索。

The final consideration is the size (number of examples) of the dataset. While there is no definite answer to how much data is enough data, algorithms always perform better when trained with huge amounts of data.

最后要考虑的是数据集的大小(示例数)。 尽管对于多少数据就是足够的数据没有确切的答案，但是当训练大量数据时，算法始终会表现更好。

The required minimum is to have at least 10times as many data examples as there are features in the dataset.

所需的最小值是数据示例的至少10倍，是数据集中存在的特征的数量 。

If this is not the case, then more data should be collected. Many options are available for getting more data. These include crowdsourcing using platforms like Amazon Mechanical Turk; other external sources; or internal data collection within the organization. For some problems, it might be possible and appropriate to generate more data from existing data examples. This is best determined by a machine learning engineer.

如果不是这种情况，则应收集更多数据。 许多选项可用于获取更多数据。 其中包括使用Amazon Mechanical Turk等平台进行众包； 其他外部来源； 或组织内部的内部数据收集。 对于某些问题，从现有数据示例中生成更多数据可能是适当的。 最好由机器学习工程师决定。

数据准备/预处理 (Data preparation/pre-processing)

At this stage, you explore the data critically and prepare or transform it such that it is ready for training. Look out for such things as missing data, duplicate examples and features, feature value ranges, the data type of values, feature units, and so on. Use easy tools to quickly examine the data and scavenge as much general information as possible. After gathering useful information, some of the following actions may be required:

在此阶段，您需要批判性地探索数据，并准备或转换数据以使其准备好进行训练。 请注意缺少数据，重复的示例和特征，特征值范围，值的数据类型，特征单位等问题。 使用简单的工具快速检查数据并清除尽可能多的常规信息。 收集有用的信息后，可能需要执行以下一些操作：

• Deal with missing data (NaN, NA, “”, ?, None) and outliers__ Standardize all missing data to np.nan. Some common options for handling missing data and outliers: dropping the data examples with missing values or applying imputation techniques (mean, mode/frequency, median).

  处理丢失的数据(NaN，NA，“”，？，无)和异常值 __将所有丢失的数据标准化为np.nan。 处理缺失数据和离群值的一些常见选项：删除具有缺失值的数据示例或应用插补技术(均值，模式/频率，中位数)。

• Deal with duplicate features and/or examples__ Duplicate features cause problems of linear dependence in the data set and duplicate examples may give a false impression of the data being enough meanwhile the number of unique examples might be too small to reasonably train a good model.

  处理重复的特征和/或示例 __重复的特征会导致数据集线性相关的问题，重复的示例可能给数据带来错误的印象，同时独特示例的数量可能太少而无法合理地训练一个好的模型。

• Feature scaling, normalization, standardization__ You want to ensure that your features are in the same or comparable ranges typically 0 to 1. This ensures that your model trains faster and is stable especially if you are using optimization algorithms like gradient descent.

  特征缩放，归一化，标准化 __您要确保特征处于相同或可比较的范围内(通常为0到1)。这可以确保模型训练更快且稳定，尤其是在使用诸如梯度下降的优化算法时。

• Balance the class sizes for categorical data__ Ensure that the number of training examples across the different target categories in your dataset is comparable. But if the task you are working on involves naturally skewed patterns where one class always dominates the other, balancing is not an option. This is common with anomaly detection tasks like rare diseases prediction (e.g. cancer), and fraud detection. An appropriate training method and evaluation metric must be chosen for skewed datasets that cannot be reasonably balanced.

  平衡分类数据的类大小 __确保数据集中不同目标类别的训练示例的数量可比。 但是，如果您正在处理的任务涉及自然偏斜的模式，其中一类总是主导另一类，那么平衡就不是一种选择。 这在异常检测任务(例如罕见病预测(例如癌症)和欺诈检测)中很常见。 必须为无法合理平衡的偏斜数据集选择适当的训练方法和评估指标。

• Harmonize inconsistent units__ Inconsistent units can easily escape notice. Ensure that all units measuring the same physical quantities are the same. Just to emphasize the point, NASA lost its $125-million Mars Climate Orbiter satellite because of inconsistent units.

  协调不一致的单元 __不一致的单元可以轻松逃脱通知。 确保所有测量相同物理量的单位都相同。 为了强调这一点， 美国宇航局由于单位不一致而损失了价值1.25亿美元的“火星气候轨道器”卫星 。

数据可视化和探索性分析 (Data visualization and exploratory analysis)

Data visualization provides the most optimum means for exploratory analysis. Using plots like histograms and scatter plots one may easily spot things like outliers, trends, clusters, or categories in your dataset. However, visualizations tend to be very useful only for low dimensional data (1D, 2D, 3D) as higher dimensions cannot be plotted. For high dimensional data, you may select some specific features to visualize.

数据可视化为探索性分析提供了最佳的方法。 使用直方图和散点图之类的图，可以轻松发现数据集中的异常值，趋势，聚类或类别。 但是，可视化仅对低维数据(1D，2D，3D)有用，因为无法绘制高维。 对于高维数据，您可以选择一些特定功能进行可视化。

特征选择和特征工程 (Feature selection and Feature engineering)

Which features are relevant to make correct predictions?

哪些特征与做出正确的预测有关？

The goal is to select features so that you have the least correlation between features but the maximum correlation between each feature and the targets.

目的是选择要素，以使要素之间的关联最少，但每个要素与目标之间的关联最大 。

Feature engineering involves manipulating the original features in the dataset into new potentially more useful features. As mentioned above, always think of what hidden features might be deduced from the original data. Debatably, feature engineering is one of the most critical and time-consuming activities in the ML pipeline.

特征工程涉及将数据集中的原始特征操纵为可能更有用的新特征。 如上所述，请始终考虑可以从原始数据中推断出哪些隐藏特征。 值得一提的是，特征工程是ML管道中最关键和最耗时的活动之一。

With all the above steps performed, you now have a sizable dataset with features that are relevant for the ML task and we can proceed (with some confidence) to train a model.

完成上述所有步骤后，您现在已经拥有一个相当大的数据集，其中包含与ML任务相关的功能，我们可以(有把握地)进行模型训练。

模型训练 (Model Training)

The first step before training is to split your dataset into a train set, cross-validation, or development set and test set with randomization.

训练之前的第一步是将您的数据集分为具有随机性的训练集，交叉验证或开发集和测试集。

Randomization helps to eliminate bias in your models and is achieved by shuffling the data before splitting. Randomization is extremely important, especially when dealing with sequential data that follows some chronological order. This will ensure that the model does not go learning the structure in the data.

随机化有助于消除模型中的偏差，可以通过在分割前对数据进行混洗来实现。 随机化非常重要，尤其是在处理遵循某些时间顺序的顺序数据时。 这将确保模型不会学习数据中的结构。

There is no guiding rule for optimum splits, but the main intuition is to have as much training data as possible; smaller but sufficient data to tune hyperparameters during training and enough test data to test the model’s ability to generalize on. Some typical and commonly used splits include:

没有最佳分割的指导规则，但主要的直觉是要拥有尽可能多的训练数据。 较小但足够的数据以在训练期间调整超参数，而足够的测试数据可测试模型的概括能力。 一些典型和常用的拆分包括：

Next, you will set aside the test set for later testing your models and proceed with the train set to train your model. It is good to quickly try out different potential algorithms and pick the one with the best generalization performance on the cross-validation or dev set for further tuning or pick a set of algorithms to form an ensemble. Use the dev set for model hyperparameter tuning.

接下来，您将预留测试集以用于以后测试模型，并继续使用训练模型来训练您的模型。 Swift尝试不同的潜在算法，并在交叉验证或开发集上选择具有最佳泛化性能的算法，以进行进一步调整，或者选择一组算法以形成整体，这是很好的。 将开发集用于模型超参数调整。

模型评估 (Model Evaluation)

Is the model useful, (does it have the minimum required performance measure)?

该模型有用吗(它具有最低要求的性能指标)吗？

Is the model computationally efficient?

模型的计算效率高吗？

Once you have optimized your model’s performance on the dev set as much as possible, you can now assess how well it performs on unseen data that was set aside in your test set. The performance observed on the test data gives you a glimpse of what you can expect to see in the production environment. Use single value evaluation metrics for quantifying performance.

一旦尽可能在开发集上优化了模型的性能，就可以评估模型在测试集中保留的未见数据上的性能。 在测试数据上观察到的性能使您可以大致了解在生产环境中可以看到的内容。 使用单值评估指标来量化性能。

• Accuracy: suitable for classification task精度：适合分类任务
• Precision/recall: suitable for skewed classification task精度/召回率：适用于倾斜的分类任务
• Rsquared: suitable for regressionRsquared：适合回归

It is hard to strike a good balance between precision and recall, hence they are always combined into a single value evaluation metric, the F1 score.

很难在精度和召回率之间取得良好的平衡，因此，它们总是组合为一个单一的价值评估指标，即F1得分 。

If the minimum required performance is obtained, then you have a useful model that is ready for deployment.

如果获得了最低要求的性能，那么您就有了一个可供部署的有用模型。

“All models are wrong, but some are useful.” __ George Box

“所有模型都是错误的，但有些是有用的。” __乔治·博克斯

模型部署，集成和监控 (Model deployment, integration, and monitoring)

“The deployment of machine learning models is the process for making your models available in production environments, where they can provide predictions to other software systems. It is only once models are deployed to production that they start adding value.” __ Christopher Samiullah

机器学习模型的部署是使模型在生产环境中可用的过程，他们可以在其中为其他软件系统提供预测。 只有将模型部署到生产中后，它们才能开始增加价值。” __克里斯托弗·萨米拉

Deployment is very crucial and probably the ML engineer’s nightmare as it is more of a software engineering discipline. Nevertheless, ML engineers are largely expected to be able to deploy and integrate their models with existing software systems to cater to end-users. I have very little to say about deployment, but a few things to note are how ML deployments fundamentally differ from explicitly programmed software.

部署非常关键，可能是ML工程师的噩梦，因为它更多地是软件工程学科。 尽管如此，人们普遍期望ML工程师能够将其模型与现有软件系统进行部署和集成，以满足最终用户的需求。 关于部署，我几乎没有什么要说的，但是要注意的是ML部署与显式编程的软件在根本上有何不同。

Models in production environments suffer from performance decay with time. As a solution, monitoring the performance of your model in production is standard practice. Performance decay is inevitable partly because of drifts in data distribution in the production environment outside the data distribution that was existent in the train set. If you notice a significant difference in the production data distribution, then you need to retrain your model.

生产环境中的模型会随着时间的推移而性能下降。 作为解决方案，在生产中监视模型的性能是标准做法。 性能下降是不可避免的，部分原因是生产环境中的数据分布在列车集中存在的数据分布之外漂移。 如果您发现生产数据分布存在显着差异，则需要重新训练模型。

Over the lifetime of any deployed ML model, the cycle monitor, retrain, and update is a routine process and it helps to use continuous logging of system performance information and creating performance drift alerts for efficient monitoring.

在任何已部署的ML模型的整个生命周期中，周期监视，重新训练和更新都是例行过程，它有助于使用系统性能信息的连续记录并创建性能漂移警报以进行有效监视。

结论 (Conclusion)

In this article, I summarized the stages of a machine learning project from understanding the problem to a usable solution.

在本文中，我总结了机器学习项目的各个阶段，从理解问题到可用的解决方案。

An ML solution is a system with a machine learning engine running in the background.

ML解决方案是一个在后台运行机器学习引擎的系统。

Summary of activities:

活动摘要：

• Understand the business problems and needs了解业务问题和需求
• Frame the ML problem框架ML问题
• Understand the data needs and acquire the data了解数据需求并获取数据
• Clean and preprocess the data清理和预处理数据
• Select relevant features选择相关功能
• Perform feature engineering执行特征工程
• Train a model训练模型
• Tune hyperparameters to optimize the performance of the model (accuracy and speed).调整超参数以优化模型的性能(准确性和速度)。
• Test the model测试模型
• Deploy the model部署模型
• Monitoring and updating the model/system (continuous process)监视和更新模型/系统(连续过程)

资源资源 (Resources)

1. Getting started with AWS machine learning course on CourseraCoursera上的AWS机器学习课程入门
2. Machine Learning course by Andrew on CourseraAndrew在Coursera上的机器学习课程

3. How to deploy machine learning models

   如何部署机器学习模型

4. 6 stages to get success in machine learning projects

   在机器学习项目中获得成功的6个阶段