I’m bothered by statements like “Clearly, the data says…” or “It is obvious from the data that…”. Most of the time, it is neither clear nor obvious. We need the reasoning. The data cannot even talk — how can it possibly speak for itself?

我对诸如“数据显然是……”或“从数据中显而易见……”这样的陈述感到不安。 在大多数情况下，它既不清晰也不明显。 我们需要推理。 数据甚至无法说话-它怎么可能自己说话？

Wait, no, let me try that. It sounds fun. Ahem. The conclusion here is clear:

等等，不，让我尝试一下。 听起来很有趣。 啊 这里的结论很清楚：

I’ve had the displeasure of coming across supremacists who cite some statistics as “proof” that certain races or sexes are inferior than others. Surely, if the “data speaks for itself”, we should be agreeing with them? And yet, many of us don’t agree. Even those without a stats background can feel something off — the statistics do not capture the bigger picture, and there are complexities (strong unmeasured confounders or mediators) that prevent us from taking the numbers at face value. Wouldn’t it be nice if we can formalize this level of healthy skepticism?

我很不高兴遇到至高无上的主义者，他们引用一些统计数据来“证明”某些种族或性别不如其他种族或性别低。 当然，如果“数据说明一切”，我们应该同意它们吗？ 但是，我们许多人不同意。 即使是没有统计背景的人也可能会感到有些不舒服-统计数据无法捕捉到更大的景象，并且由于存在复杂性(强大的无法衡量的混杂因素或调解人)，我们无法以实际价值来衡量数字。 如果我们可以将这种健康怀疑论的形式正式化，那不是很好吗？

For the longest time, I had trouble articulating this idea until I came across Cassie’s article. She states:

在最长的时间内，直到我碰到卡西的文章时，我都很难阐明这个想法。 她说：

INFERENCE = DATA + ASSUMPTIONS

推论=数据+假设

It is our assumptions that give voice to the data. I want to decompose this further, because there is a subtle but important distinction:

我们的假设使数据具有说服力。 我想进一步分解，因为有一个微妙但重要的区别：

Inference = Data + Statistical Assumptions + Causal Assumptions

推论=数据+统计假设+因果假设

Why the split? Even with single parameter estimates like “the average height of the population is X” or “there are Y trees in the forest”, we need to assume some data generation mechanism like simple random sampling — a causal statement.

为什么要分裂？ 即使使用诸如“人口的平均高度为X”或“森林中有Y棵树木”之类的单参数估计，我们也需要假设一些数据生成机制，例如简单的随机抽样-因果关系陈述。

The data scientists / data analysts might be experts in the statistical assumptions, but they might not know much about the causal assumptions. For instance, if you are a statistician who works with medical data, you’d be crazy to think that you know more about medicine than the MD or MPH. Some are experts in both the statistics and the scientific domain, but they are rare.

数据科学家/数据分析师可能是统计假设方面的专家，但他们可能对因果假设了解不多。 例如，如果您是一位处理医学数据的统计学家，那么您会以为自己比MD或MPH更了解医学，这会让您发疯。 有些是统计学和科学领域的专家，但很少见。

The quality of assumptions greatly influences the quality of insights. Yet, people rarely talk about assumptions. Teams would do well to hold meetings to discuss them.

假设的质量极大地影响了见解的质量。 但是，人们很少谈论假设。 团队最好召开会议来讨论它们。

Working in inference means setting aside your ego. You have to show humility and defer to others’ expertise so you can incorporate their knowledge into the model. It is a team effort.

进行推理意味着抛开自我。 您必须表现出谦卑，并尊重他人的专业知识，以便将他们的知识整合到模型中。 这是团队的努力。

辛普森悖论 (Simpson’s Paradox)

The UCB Admissions dataset is one of the simplest and best examples to illustrate the importance of causal assumptions. You might know it from Simpson’s Paradox. The data (column names not mine):

UCB入学数据集是说明因果假设重要性的最简单，最好的例子之一。 您可能会从Simpson的悖论中了解到这一点。 数据(列名不是我的)：

The question: is there sexism in UCB’s admission process?

问题： UCB的录取过程中是否存在性别歧视？

Anna crunches some numbers. She adds up the counts, like so:

安娜处理一些数字。 她将计数加起来，如下所示：

And runs logistic regression:

并运行逻辑回归：

Anna concludes “The coefficient for female is negative with a small p-value. Clearly, the data says that the admissions process is sexist.”

安娜得出结论：“女性系数为负，p值较小。 显然，数据表明录取过程是性别歧视。 ”

Barbara also crunched the numbers, but she kept the original data as is. She runs logistic regression, controlling for Dept:

芭芭拉(Barbara)还计算了数字，但她保留了原始数据。 她进行逻辑回归，控制部门：

Barbara concludes “The coefficient for female is positive with a large p-value. Clearly, the data says that the admissions process is not sexist.”

芭芭拉得出结论：“女性系数为正，且p值较大。 显然，数据表明录取过程并非性别歧视。 ”

等一下 (Wait, what?)

They both use the same data. They both have reasonable statistical assumptions. Yet they come to wildly different conclusions because of different methodology. Who’s right?

它们都使用相同的数据。 他们都有合理的统计假设。 但是由于方法不同，他们得出了截然不同的结论。 谁是对的？

I reiterate what I wish were taught in stats courses:

我重申希望在统计课程中教授的内容：

Data + statistical assumptions are not enough to draw inference. We need causal assumptions.

数据+统计假设不足以进行推断。 我们需要因果假设。

So how should we approach this? To simplify, assume that we have measured all the relevant variables. We lay out some causal assumptions:

那么我们应该如何处理呢？ 为了简化，假设我们已经测量了所有相关变量。 我们提出一些因果假设：

1. Gender directly affects admission rate (sexism)性别直接影响入学率(性别歧视)
2. Dept directly affects admission rate (some departments are harder to get into)系直接影响录取率(某些部门更难进入)
3. Gender directly affects Dept (male applicants are drawn to different majors than female applicants)性别直接影响部门(男申请者被吸引到与女申请者不同的专业)

(1) is what we want to test. For formality, we assume (1) is true so that we can test it by putting it into the model.

(1)是我们要测试的。 为了形式，我们假设(1)是正确的，以便我们可以将其放入模型中进行测试。

What about (2) and (3)? If you ask me, both are reasonable assumptions, and we can assume them to be true.

那(2)和(3)呢？ 如果您问我，这都是合理的假设，我们可以假设它们是正确的。

“Whoa, hold your horses!” some might say. “That is mighty uncomfortable! You can’t just assume those things!” But, but, but! We must make those choices; we cannot be agnostic. When I first learned about causal modeling, I had the same reservations and eventually warmed up to the idea. For any given pair of variables X and Y, we must pick one of:

“哇，抱着你的马！” 有些人可能会说。 “那真让人不舒服！ 你不能只是假设那些事情！” 但是，但是，但是！ 我们必须做出这些选择； 我们不可能是不可知论的。 当我第一次了解因果建模时，我有相同的保留意见，并最终热衷于这个想法。 对于任何给定的变量X和Y对，我们必须选择以下一项：

• X directly influences YX直接影响Y
• Y directly influences XY直接影响X
• X and Y do not directly affect each otherX和Y不会直接相互影响

In the case of (2), “admission rate influences department” doesn’t make sense while “department and admission rate are not directly related” is certainly false (depending on the other assumptions, it can mean “all departments have the same admission rate”). The only plausible thing that’s left is “department influences admission rate”.

对于(2)，“入学率对部门的影响”没有意义，而“部门与入学率没有直接关系”肯定是错误的(根据其他假设，这可能意味着“所有部门具有相同的入学率”率”)。 唯一剩下的似乎是“部门影响录取率”。

The choice of assumptions are implied by the analyst’s methodology even if not verbally stated, though one methodology can correspond to multiple sets of assumptions. Just because they’re unsaid does not mean those assumptions don’t exist. If anything, the unstated assumptions are often the ones that make the least sense (see my previous article). It’s almost like lying by omission for the sake of appearing “objective”.

分析人员的方法论隐含了假设的选择，即使没有口头说明也是如此，尽管一种方法论可以对应于多组假设。 只说不说并不意味着这些假设不存在。 如果有的话，未陈述的假设通常是最没有意义的假设(请参阅我的上一篇文章 )。 这几乎就像是为了表现出“客观”而疏忽撒谎。

The three assumptions taken together imply this causal diagram:

这三个假设合起来意味着此因果关系图：

Why go through all this trouble? If you are not familiar with causal diagrams, I suggest reading the last third of my article on causality. As it turns out, the assumptions tell us how we should analyze the data:

为什么要经历所有这些麻烦？ 如果您不熟悉因果关系图，建议您阅读文章中有关因果关系的最后三分之一。 事实证明，这些假设告诉我们如何分析数据：

• (1) or (1) + (3) dictate that we aggregate all the data and use Gender as a predictor(1)或(1)+(3)指示我们汇总所有数据，并使用Gender作为预测变量
• (1) + (2) dictate that we use the raw data and use Gender as a predictor(1)+(2)规定我们使用原始数据，并使用Gender作为预测变量

• (1) + (2) + (3) dictate that we use the raw data and use Gender and Dept as predictors

  (1)+(2)+(3)指示我们使用原始数据，并使用性别和部门作为预测变量

Anna’s methodology assumes that departments and admission rate are not directly related. I highly doubt that, so her analysis is faulty. (Be careful. Even faulty analyses can yield correct conclusions by accident.)

Anna的方法假设部门和录取率没有直接关系。 我对此表示高度怀疑，因此她的分析是错误的。 (请注意。即使是错误的分析也可能会偶然得出正确的结论。)

Barbara’s methodology assumes (2) and (3). This is much more reasonable than Anna’s assumptions, so I’m more inclined to believe in her conclusion that the admission process is not sexist.

芭芭拉的方法假设(2)和(3)。 这比安娜的假设更为合理，因此我更倾向于相信她的结论，即录取过程并非性别歧视。

因果模型 (Causal modeling)

“Controlling for X” is the same as “slicing and dicing by X”, except your variables do not have to be categorical.

“控制X”与“按X切片和切块”相同，不同之处在于您不必对变量进行分类。

Causal modeling is the art of slicing and dicing data the “right” way.

因果建模是一种以“正确”方式对数据进行切片和切块的艺术。

FiveThirtyEight made a fun interactive page where you can p-hack your way into statistical significance. Give it a try. If you regress Employment Rate on Governors, Republicans have a positive effect on the economy (p < 0.01). If you regress Employment Rate on Presidents, Republicans have a negative effect on the economy (p = 0.02). So what does the data say? Don’t confuse modeling choices with an inherent quality of the data.

FiveThirtyEight创建了一个有趣的交互式页面 ，您可以在其中p破解自己的统计意义。 试试看。 如果将州长的就业率降低，共和党人会对经济产生积极影响(p <0.01)。 如果将总统的就业率降低，共和党人会对经济产生负面影响(p = 0.02)。 那么数据怎么说呢？ 不要将建模选择与数据的固有质量混淆。

An old joke goes “What do you get when you put 10 economists in a room? 11 opinions.” A more accurate joke would be “What do you get when you put 10 statisticians in a room? 10 estimates.”

一个古老的笑话说：“当你把10个经济学家放在一个房间里时，你会得到什么？ 11条意见。” 一个更准确的笑话是“将10位统计学家放在一个房间里，您会得到什么？ 10个估算值。”

And therein lies the problem. People get different conclusions depending on how they choose to slice and dice the data. What is even the “correct” conclusion? Does it exist? Is it a mythical creature? A Platonic ideal?

问题就在这里。 人们根据如何选择对数据进行切片和切块而得出不同的结论。 什么是“正确”结论？ 是否存在？ 这是一个神话生物吗？ 柏拉图式的理想？

Instead of arguing about our different conclusions, how about discussing causal assumptions? Perhaps we can get the entire team to agree on one way to slice and dice the data. And, hopefully, all parties involved have enough grace to accept the conclusion that comes from this single way of slicing and dicing. This, in essence, is what causal modeling is.

与其争论我们的不同结论，不如讨论因果假设？ 也许我们可以让整个团队就一种将数据切片和切块的方法达成共识。 并且，希望所有参与方都具有足够的宽限度，可以接受来自这种单一切片和切块方法的结论。 本质上，这就是因果建模。

A/B tests are valuable because that’s the only time we fully know the causal diagram (caveat: not true if you have missing data, or if there is effect modification from other partly overlapping experiments, or if novelty effect is strong, or…). It is the only case where we do not have to make causal assumptions. We know exactly how to slice and dice the data correctly. The problem reduces purely to statistical assumptions.

A / B测试非常有价值，因为这是我们唯一完全了解因果关系图的情况(警告：如果您缺少数据，或者其他部分重叠的实验对效果进行了修改，或者新颖性效果很强，则为……) 。 这是唯一我们不必进行因果假设的情况。 我们确切地知道如何正确地对数据进行切片和切块。 这个问题纯粹是出于统计假设。

Of course, there are plenty of issues with causal modeling, such as this criticism. Yet, despite all its flaws and limitations, causal modeling is the best tool we’ve got.

当然，因果建模存在很多问题，例如这种批评 。 然而，尽管存在所有缺陷和局限性，因果建模还是我们拥有的最佳工具。

这会让你成为一个男人吗？ (Will this make you a yes-man?)

In the spirit of Betteridge’s Law: no.

本着贝特里奇定律的精神： 不可以 。

In the UCB Admissions example, we can collectively agree on the assumptions (1) + (2) + (3) and yet someone objects to the results. We can ask: which assumption(s) do you have an issue with?

在UCB入学示例中，我们可以集体同意假设(1)+(2)+(3)，但有人反对结果。 我们可以问：您对哪个假设有疑问？

For example, if they disagree with (3), as long as the team as a whole acts in good faith, other team members should chime up and say that (3) is the most plausible option. Assumptions are a group effort.

例如，如果他们不同意(3)，则只要整个团队以诚实的态度行事，其他团队成员就应该保持警惕，并说(3)是最合理的选择。 假设是集体努力。

If they agree with all the causal assumptions and still object, then perhaps the issue is with the statistical assumptions. This is an entire article on its own, so I’ll skip it.

如果他们同意所有因果假设并且仍然反对，那么问题可能出在统计假设上。 这是一篇完整的文章，因此我将跳过。

If they agree with all the causal and statistical assumptions but still doubt the conclusion, well, it’s easier for the team to see that this objection is irrational.

如果他们同意所有因果假设和统计假设，但仍对结论感到怀疑，那么，团队更容易看出这一反对意见是不合理的。

And, just like how experiments can be improved by making them double-blind, the team can discuss the assumptions without anyone knowing what the result of those assumptions will be.

并且，就像如何通过使实验成为双盲来改进实验一样，团队可以在不知道这些假设的结果的情况下讨论这些假设。

I wonder how much happier and more collaborative the teams will be when assumption-driven analytics becomes the norm? Everyone has a say in the analysis instead of leaving it only to the stats folks, and by agreeing on the terms upfront, the outcomes and decisions will be judged as more fair.

我想知道当假设驱动的分析成为规范时，团队将有多快乐和合作？ 在分析中，每个人都有发言权，而不仅仅是将其留给统计人员，并且通过预先约定条件，结果和决定将被认为更加公平。

预测建模被高估 (Predictive modeling is overrated)

There, I said it.

在那里，我说了。

I always tell people, before working on anything, to ask themselves: prediction or inference? You cannot have both. (Well, okay, you can, but only sometimes, as a treat.) Unfortunately, many people are not even aware of the distinction because most of data science is concerned with prediction.

在做任何事情之前，我总是告诉人们问自己：预测还是推论？ 你们不能两者兼得 。 (嗯，好的，您可以，但仅在某些情况下可以作为一种对待。)不幸的是，许多人甚至没有意识到这种区别，因为大多数数据科学都与预测有关。

For many applications, we only care about prediction — think putting models into production. The only conclusion we need to draw is “does this model make good predictions?” This is a privilege.

对于许多应用程序，我们只关心预测-考虑将模型投入生产。 我们需要得出的唯一结论是“该模型能否做出正确的预测？” 这是特权。

However, decision making from data requires inference. Drawing conclusion from data requires inference. Recommending action from data requires inference.

但是，根据数据进行决策需要推理。 从数据中得出结论需要推理。 从数据推荐操作需要推理。

Naturally, purely predictive models are bad for those tasks. Consider models like random forest with no semblance of interpretability other than LIME, which makes no statement outside a locale. What action should you take based on the variable importance plot? How does it aid decision making in any way?

自然地，纯粹的预测模型对那些任务不利。 考虑像LIFE这样的模型，除了LIME之外没有其他可解释性，它不会在语言环境外声明。 根据可变重要性图应采取什么措施？ 它如何以任何方式帮助决策？

Even if you use a more interpretable model like elasticnet, you cannot throw in everything, maximize predictive power, and expect to get good inferences. You might be conditioning on a collider and get spurious correlation, or you might be blocking a causal pathway and hide the effect (if you don’t understand what this means, please read the last third of my article on causality). An illustrative quote:

即使您使用诸如Elasticnet之类的更具可解释性的模型，您也无法投入一切，无法最大化预测能力，并期望获得良好的推论。 您可能正在对撞机进行调节并获得虚假的相关性，或者您可能正在阻止因果关系并隐藏其影响(如果您不了解这是什么意思，请阅读我的因果关系文章的后三分之一)。 一个说明性报价：

Controlling for barometric pressure, Mount Everest has the same altitude as the Dead Sea. (source)

通过控制气压，珠穆朗玛峰的海拔高度与死海相同。 ( 来源 )

Surely that’s a silly conclusion, but people unknowingly draw that kind of conclusion all the time by “controlling for everything”. It’s rarely obvious. Or, equally bad, people conclude that turning sprinklers on will prevent rain. (Hey, it is an excellent predictor.)

当然，这是一个愚蠢的结论，但是人们总是在不知不觉中通过“控制一切”得出这样的结论。 这很少见。 或者，同样糟糕的是，人们得出的结论是，打开喷头可以防止降雨。 (嘿，它是一个很好的预测指标。)

Causal modeling is a case of “we have the product that customers (business stakeholders) need, but the customers don’t know that this is what they need”.

因果模型是“我们拥有客户(业务利益相关者)需要的产品，但客户不知道这就是他们所需要的”的情况。

“Actionable insights” has become some kind of a buzzword in data science. Yet, people chase purely predictive models that cannot possibly yield “actionable insights” while they ignore inferential / causal modeling. It is strange. Causal modeling should be the heart of analytics.

“可行的见解”已成为数据科学中的一种流行语。 但是，人们追求纯粹的预测模型，而忽略了推论/因果模型，就无法产生“可行的见解”。 它很奇怪。 因果模型应该是分析的核心。

在结束时 (In closing)

If someone tries to browbeat others with data by saying “The data clearly says X!”, try asking them what their causal assumptions are. It’s illuminating. If they claim there are no assumptions, either the data comes from an A/B test or it might as well be:

如果某人试图通过说“数据清楚地用X！”来殴打他人，请尝试询问他们的因果假设是什么。 照亮了 如果他们声称没有假设，那么数据要么来自A / B测试，要么可能是：

The more I work with data, the less certain I get about conclusions. The ones parading around a statistic like it is a glimpse of ThE oNe TrUtH might very well be under the Dunning-Kruger Effect: they make strong inferential statements without knowing the role that assumptions play in inference.

我处理数据越多，得出结论的把握就越少。 像这样一则关于统计数据的概述很可能是邓恩 -克鲁格效应的 一种概括 ：他们在不知道假设在推论中扮演什么角色的情况下做出了强有力的推论陈述。

We rarely know whether or not we are slicing and dicing the data the “right” way, though some ways are more believable than others. As much as we like to think that science and faith are polar opposites, much of science rests on unverifiable assumptions that require a leap of faith. Always be open to the possibility of being wrong.

尽管有些方法比其他方法更可信，但我们很少知道我们是否以“正确”的方式对数据进行切片和切块。 尽管我们喜欢认为科学和信仰是两极对立，但许多科学都建立在不可验证的假设之上，这些假设需要信念的飞跃。 始终对犯错的可能性持开放态度。