Statistical modeling is a lot like engineering.

In engineering, there are various ways to build a key-value storage, and each design makes a different set of assumptions about the usage pattern.

In statistical modeling, there are various algorithms to build a classifier, and each algorithm makes a different set of assumptions about the data.

When dealing with small amounts of data, it’s reasonable to try as many algorithms as possible and to pick the best one since the cost of experimentation is low.

But as we hit “big data”, it pays off to analyze the data upfront and then design the modeling pipeline(pre-processing, modeling, optimization algorithm, evaluation,product ionization) accordingly.

As pointed out in my previous post,there are dozens of ways to solve a given modeling problem.

Each model assumes something different, and it’s not obvious how to navigate and identify which assumptions are reasonable.

In industry, most practitioners pick the modeling algorithm they are most familiar with rather than pick the one which best suits the data.

In this post, I would like to share some common mistakes (the don't-s).

I’ll save some of the best practices (the do-s) in a future post.

前言部分，作者提出了两个现象：

1.现在数据运算的成本很低了，大家都是直接所有方法都试一遍。

2.人们更倾向于使用自己熟悉的方法而不是最适合需要处理的数据的方法。

写到这里，即使如我这种屌丝都看出作者是不赞成这样的。下文就开始分析7大常犯错误。

1. Take default loss function for granted 想当然用缺省Loss

Many practitioners train and pick the best model using the default loss function (e.g., squared error).

In practice, off-the-shelf loss function rarely aligns with the business objective.

Take fraud detection as an example. When trying to detect fraudulent transactions, the business objective is to minimize the fraud loss.

The off-the-shelf loss function of binary classifiers weighs false positives and false negatives equally.

To align with the business objective, the loss function should not only penalize false negatives more than false positives, but also penalize each false negative in proportion to the dollar amount.

Also, data sets in fraud detection usually contain highly imbalanced labels.

In these cases, bias the loss function in favor of the rare case (e.g., through up/down sampling).

“机器学习本质上是在解一个优化问题，优化目标定义错误（或者 loss function 定义错了），就全错了！”

作者在这里以欺诈交易识别为例，说明了常用的损失函数(例如误差的平方和)是不适用的。

2. Use plain linear models for non-linear interaction 非线性情况下用线性模型