蠕变断裂 ansys

by Dror Berel

由Dror Berel

如何避免范围蠕变，以及其他软件设计课程的辛苦学习方法 (How to avoid scope creep, and other software design lessons learned the hard way)

从数据科学的角度来看。 (From a data-science perspective.)

You’ve got a fresh new project on your desk, some exciting data, a challenging Kaggle competition, a new client you wish to impress, and you are fully motivated. At first, the problem seems to be well defined, and you even feel comfortable with the task in hand. You have just completed a similar task. This new one should not be much different. Maybe even just a few copy/pastes with some modifications at the edges.

您的办公桌上有一个崭新的项目，一些令人兴奋的数据，充满挑战性的Kaggle竞赛，您希望给新客户留下深刻的印象，并且您充满了动力。 最初，问题似乎已经很好地定义了，您甚至对手头的任务感到满意。 您刚刚完成了类似的任务。 这个新的应该没有太大的不同。 甚至可能只是一些复制/粘贴，并且在边缘进行了一些修改。

But then it comes… The client / collaborator / boss has just one simple additional request… It usually goes like this:

但是随之而来……客户/合作者/老板只有一个简单的附加请求……通常是这样的：

‘Hmmmm, I wonder how would the results look like if instead of x, we do only a minor change, just do y, or… you know what, let’s try both and see how it affects the results’.

“嗯，我想知道结果是什么样的，如果我们只做一个微小的改变而不是x，或者做y，或者……您知道吗，让我们同时尝试一下，看看它如何影响结果。”

Can the initial tool/solution you chose handle such an adjustment? It may be easy to copy/paste it with a couple of alterations, but what if you have to do it again and again? For how long are you going to stick to your initial plan?

您选择的初始工具/解决方案可以处理这种调整吗？ 复制/粘贴一些更改可能很容易，但是如果必须一遍又一遍怎么办？ 您将坚持最初的计划多长时间？

Within the context of machine learning, some examples are:

在机器学习的上下文中，一些示例是：

Tuning ‘let’s see how a different model parameter affects it’

调整 “ 让我们看看不同的模型参数如何影响它”

Benchmarking ‘let’s see how various models affect it’

基准测试 “ 让我们看看各种模型如何影响它”

Ensemble ‘let’s try combining the best models together’

合奏 “ 让我们尝试将最佳模型组合在一起 ”

Resampling / cross-validation ‘we must inspect for over-fitting’

重新采样/交叉验证 “ 我们必须检查是否过拟合 ”

Imagine adding on top of that some complex, messy, multi-layer, high-throughput genomics data that can easily go into a very fine resolution level (gene expression / mutation / sequence, …),… AND THEN adding multiple layers of various multi-genomic data on top of each other, …AND THEN doing it for multiple cohorts / studies in a meta-analysis level … you may end up with a VERY … BIG … UGLY … MESS!

想象在这些复杂，混乱，多层，高通量的基因组数据的基础上添加，这些数据可以轻松进入非常精细的分辨率级别 (基因表达/突变/序列等)，然后再添加多层多层-基因组数据互为基础，...然后在荟萃分析级别将其用于多个群组/研究中...您可能最终会感到……非常……非常……非常……非常！

Sound familiar? Unfortunately, I have been in this situation more than once. As much as I was motivated to please my collaborators, at those times, my tools were…limited, and not sufficient to deliver the broader scope resolution. At that time, I might have not even been aware that a higher level of scope was relevant.

听起来有点熟？ 不幸的是，我不止一次遇到这种情况。 尽我所能去取悦我的合作者，当时我的工具是……有限的，不足以提供更广泛的解决方案。 那时，我什至可能还没有意识到更高层次的范围是有意义的。

A lot has been written about scope creep in the context of project management. But what would a scientist, who was mostly trained to care about the rightness of the analysis / tools, rather than the ‘management’ of the whole project, have to say about it?

在项目管理的背景下，有关范围蠕动的文章很多。 但是，一位受过最多培训的科学家关心的分析/工具的正确性，而不是整个项目的“管理”，该怎么说呢？

The good news, my friend, is that it is never too late to learn from someone else’s mistakes. Here are couple of lessons, learned the hard way. (No worries, this is not another blog post about reproducible research).

我的朋友，好消息是，从别人的错误中学习永远不会太晚。 这是几节课，是艰难学习的方法。 (不用担心，这不是有关可重复研究的另一篇博客文章)。

第一课：从头开始！ 定义您的范围。 您需要扩展它吗？ (Lesson #1: Begin at the end! Define what your scope is. Do you need to extend it?)

Make sure you understand what is the highest expected resolution! Brainstorm what would be the craziest outcomes of your project, and then agree on reasonable expectations within your timeframe and budget.

确保您了解最高的预期分辨率！ 集体讨论您的项目最疯狂的结果，然后在您的时间范围和预算内就合理的期望达成共识。

Have a very detailed, clear, definition of the project scope. For example, is your solution going to handle just one data set, or more? How are you going to validate your results? There are always going to be more methods/data sets for that, but what would be just sufficient enough?

对项目范围有一个非常详细，清晰的定义。 例如，您的解决方案是否仅处理一个或多个数据集？ 您将如何验证结果？ 总是会有更多的方法/数据集，但是什么才足够？

The tricky challenge with scope creep is that the client doesn’t really care or think in terms of “scope”. Their goal is to get a solution that solves a hypothesis, or a business need. Whether their request is within or outside scope is entirely your problem! DEAL WITH IT!

范围蔓延的棘手挑战是客户并不真正在乎“范围”。 他们的目标是获得解决假设或业务需求的解决方案。 他们的要求是在范围内还是范围外完全是您的问题！ 处理它！

In the context of machine learning, back in the day, I used ad-hoc R packages that do just one multivariate model. They did the work well, but were too specific for the developers domain, and lacked the higher resolution on comparing it with other models, or aggregating other models, or lacking resample implementation. Only later did I learn to utilized machine learning meta/aggregator packages such as mlr, tidymodels (formerly caret), or SuperLearner to extend my scope. Read more about it here.

在过去的机器学习中，我使用了只做一个多元模型的即席R软件包。 他们的工作做得很好，但是对于开发人员领域而言过于具体，并且在与其他模型进行比较，将其他模型进行汇总或缺少重新采样实现方面缺乏更高的分辨率。 直到后来，我才学会利用机器学习元/聚合器程序包(例如mlr，tidymodels(以前为插入号)或SuperLearner)来扩展我的范围。 在此处了解更多信息 。

第2课：不要重新发明轮子！ 还有其他专家比您知道如何做得更好！ (Lesson #2: Do not reinvent the wheel! There are other experts that know how to do it better than you!)

In a role where you are expected to be multidisciplinary, and new tools/methods pop daily that are accessible for everyone to use, it may be a slippery fall into a very deep rabbit hole to explore any new approach. And guess what, nobody want you to waste their time/money on that.

在这个角色中，您应该是多学科的，并且每天都有新的工具/方法可供所有人使用，这可能是一个滑入非常深的兔子洞以探索任何新方法的方法。 猜猜是什么，没人希望您在此浪费时间/金钱。

How to bet on the right tool? Ask yourself, what do the experts in that domain use? How mature is the tool they developed? Is it going to be maintained, or deprecated? They of course had their own learning curve, and over time, have perfected their tools to overcome the common pitfalls you are about to discover.

如何下注正确的工具？ 问问自己，该领域的专家使用了什么？ 他们开发的工具有多成熟？ 是要维护还是不推荐使用？ 他们当然有自己的学习曲线，并且随着时间的推移，他们已经完善了他们的工具，以克服您将要发现的常见陷阱。

For me, with genomics data, it was Bioconductor Object-oriented S4 classes. Read more here about why that was the best tool for my need. Sure, it wasn’t trivial to learn, but I felt comfortable betting on it when I saw how it is implemented at top academic and industry organizations. I also knew that it was not another open source resource that might die. Instead, it as a government and academia-funded project, powered by the best experts of the domain, open, and free, for all of us to use.

对我而言，利用基因组学数据，它是Bioconductor 面向对象的S4类。 在这里关于为什么那是我需要的最好工具。 当然，学习并不是一件容易的事，但是当我看到它在顶级学术和行业组织中是如何实现的时，我感到很放心。 我还知道，这可能不是另一个开源资源。 相反，它是由政府和学术界资助的项目，由该领域的最佳专家提供支持，开放，免费，供我们所有人使用。

第3课：发现差距了？ 要有创造力，但要保持简单！ (Lesson #3: Found a gap? Be creative, but keep it simple!)

But what if something in the analytical pipeline is still not in place? A missing link, nowhere to be found, that would have better fit to the specific need you have, bridging the gap?

但是，如果分析管道中的某些内容仍然不存在怎么办？ 找不到链接的缺失链接会更好地满足您的特定需求，从而弥合差距吗？

Here you might need to get some dirty work done, and stop depending on others to provide you the solution. Another potentially slippery scope creep rabbit hole? Maybe… if you are not careful enough!

在这里，您可能需要完成一些肮脏的工作，然后停止其他工作来为您提供解决方案。 另一个可能滑的示波器蠕变兔子洞？ 也许……如果您不够小心！

How to avoid it? Very easy: Keep it Simple!

如何避免呢？ 非常简单： 保持简单！

Here is a very simple example. Suppose you have to solve an unsupervised problem. There is definitely more than one way to it. Which one to choose? Is the simplest one, suppose ‘hierarchical clustering’, just be good enough to begin with? Implement it, see how it works with the rest of your analytical components (data, scalability, reproducibility), and later on, after things have worked out well as you planned, relax that simplification into a more complex method. Do it very carefully and gradually.

这是一个非常简单的示例。 假设您必须解决一个无监督的问题。 肯定有不止一种方法。 选择哪一个？ 最简单的假设是“层次集群”吗？ 实施它，查看它如何与其余分析组件(数据，可伸缩性，可再现性)一起使用，然后在按计划进行一切工作之后，将其简化为更复杂的方法。 请非常小心并逐步进行。

More examples to follow next.

接下来将有更多示例。

第4课：不要害怕重构！ (Lesson #4: Do not be afraid to refactor!)

Tired of patching and debugging poorly cohesive and poorly-designed code that someone else, maybe even your boss, has written long time ago, before better tools became available? You ask yourself, GRRRRR, this is such an ugly workaround, why not just simply use that new approach that was designed specifically for this task? (see lesson #2).

厌倦了修补和调试缺乏凝聚力和设计欠佳的代码，而这些代码是别人(甚至是您的老板)很久以前在更好的工具可用之前编写的？ 您问自己，GRRRRR，这是一个丑陋的解决方法，为什么不仅仅使用专门为此任务设计的新方法呢？ (请参阅第2课)。

Yes, it is risky to begin everything from scratch, and sometimes you may not have the resources to do it, but perhaps it is time for a reality check.

是的，从头开始一切都是冒险的，有时您可能没有足够的资源来做，但是也许是时候进行现实检查了。

But what if the refactor solution will give us different results from what our collaborators are already counting on? Well, if there was indeed a past error/bug/mistake, it is better to face it and acknowledge it now, before even more damage is done. But also remember lesson #3: If you stick to simple solutions at the core, refactoring them under broader wrapping solution should assist in producing similar results.

但是，如果重构解决方案将给我们带来不同于我们合作者已经期望的结果的结果呢？ 好吧，如果确实存在过去的错误/错误/错误，最好在进一步造成损害之前立即面对并确认它。 但也要记住第3课：如果您坚持使用简单的解决方案作为核心，那么在更广泛的包装解决方案中对其进行重构应有助于产生相似的结果。

第5课：进入第1课。 (Lesson #5: go to lesson #1.)

实例探究： (Case studies:)

Here are two case studies from my own experience working with multi genomic data. (Could easily expand to other types of data, but perhaps that is a topic for a future post).

这是根据我自己在处理多基因组数据方面的经验得出的两个案例研究。 (可以轻松扩展到其他类型的数据，但这也许是以后发布的主题)。

Case Study #1: Bioc2mlr: A utility function to transform Bioconductor’s S4 omic classes into mlr’s task and CPOs.

案例研究1：Bioc2mlr：一种实用程序功能，可将Bioconductor的S4 omic类转换为mlr的任务和CPO。

https://drorberel.github.io/Bioc2mlr/

https://drorberel.github.io/Bioc2mlr/

I love using Bioconductor data containers for genomic data, but I also love machine learning meta-aggregator toolkits for analysis at higher level scope. The only problem was that they were not necessarily compatible with each other.

我喜欢将Bioconductor数据容器用于基因组数据 ，但我也喜欢机器学习元聚合器工具包 ，可以在更高层次上进行分析。 唯一的问题是它们不一定彼此兼容。

The S4 object oriented had multiple dimensions (slots), tied in together in complex constraints, that were intentionally designed to meet some purpose. But the machine learning approach was designed for a simplified, flat, two-dimensional, matrix like input structure: columns for the features/variables, and rows for the subjects/observations.

面向S4的对象具有多个维度(插槽)，它们在复杂的约束条件下捆绑在一起，这是有意设计的，可以满足某些目的。 但是，机器学习方法是为简化的，扁平的，二维的，矩阵状的输入结构设计的：特征/变量的列和主题/观察值的行。

I needed some way of breaking the S4 constrained ties, and flattening it. But unfortunately, to the best of my knowledge, I couldn’t find a way to do so. What should I have done?

我需要一些打破S4约束关系并将其展平的方法。 但不幸的是，据我所知，我找不到办法。 我该怎么办？

Remember lesson #3: Should I spend my time on this task? Well… yes, why not? I felt comfortable enough with both approaches, have already experienced the ins and outs, the soft bellies, and I definitely appreciated the tremendous value of both approaches separately, but also jointly. In fact, creating this adapter package, Bioc2mlr, was not too much effort to do, and if you look at the code itself, you will see relatively simple steps.

记住第3课：我应该花时间在此任务上吗？ 好吧，是的，为什么不呢？ 我对这两种方法都感到很自在，已经体验过前后的动作，轻柔的腹部，并且我肯定会分别和联合使用这两种方法的巨大价值。 实际上，创建此适配器包Bioc2mlr并不需要花费太多精力，如果您看一下代码本身，将会看到相对简单的步骤。

Conclusion of case 1: When you have a couple of good tools, but they are not compatible, create a simple new adapter to link them.

案例1的结论 ：如果您有几个不错的工具，但是它们不兼容，请创建一个简单的新适配器来链接它们。

Case study #2: meta analysis

案例研究2：元分析

But that wasn’t enough for me…(see lesson #5).

但这对我来说还不够……(请参阅第5课)。

My scope extension required me to provide a solution to even higher level of analysis. Meta-analysis of multiple studies/cohorts, each with a multi-omic data cube, each with a downstream machine learning analytics pipeline, implementing resampling, and all that jazz, across all studies, and at scale. Phewww!

我的范围扩展要求我提供更高级别分析的解决方案。 对所有研究/群组的荟萃分析，每个研究/群组均具有多组数据立方体，每个研究/群组均具有下游机器学习分析管道，实施重采样以及所有研究中的所有爵士乐。 哎呀！

Quite a challenge! How should I address that implementing the above lessons?

挑战很大！ 我应该如何解决实施上述课程？

Lesson #1: I began at the end. My ‘observation-unit’, row, in a tidy-fashion is not the subject, neither is the gene, nor is just one of the omics. It is the entire study/cohort (that is, a whole data cube) well-compressed into a single object in R. More than one cohort? Not a problem at all. Add as many as rows as you need for more cohorts.

教训＃1：我从结尾开始。 我的“观察单位”整齐划一，既不是主题，也不是基因，也不是组学之一。 它是将整个研究/队列(即整个数据立方体)很好地压缩到R中的单个对象中。多个队列吗？ 完全没有问题。 根据需要添加尽可能多的行，以获取更多同类。

Lesson #2: Didn’t have to invent a new tool. The experts in our field have already figured it out for us. They might have not had this implementation in mind when they did so, but if I can do it, so can you. Just give it a try.

第二课：不必发明新工具。 我们领域的专家已经为我们找到了解决方案。 他们这样做时可能没有想到这种实现，但是如果我能做到，那么您也可以。 试一试。

Lesson #3: I found a simple solution. Should I invent/extend a new S4 object oriented class for this type of multi-cohort, multi-omic data? Of course not. There must be a simple solution. My simple solution: a tidy / nested data structure, with non-atomic objects at each cell. Read more about it here.

第3课：我找到了一个简单的解决方案。 我是否应该为这种多队列，多组数据类型发明/扩展一个新的面向对象的S4类？ 当然不是。 必须有一个简单的解决方案。 我的简单解决方案：整洁/嵌套的数据结构，每个单元格都具有非原子对象。 在此处了解更多信息 。

Lesson #4: Refactor? Well. Maybe I am not there yet now, since so far my (current) scope can handle all my wildest dreams. But if you show me a better approach, perhaps a data.table one (I know), or even in python (god forbid), I would not hesitate to give it a try, even if it is beyond my comfort zone.

第4课：重构？ 好。 也许我现在还不在那里，因为到目前为止，我(当前)的范围可以处理我所有最疯狂的梦想。 但是，如果您向我展示了一种更好的方法，也许是一个data.table(我知道)，甚至是python(上帝禁止)，我都会毫不犹豫地尝试一下，即使它超出了我的舒适范围。

Lesson #5: Meta-meta analysis? (Not a typo). Who knows. Maybe one day.

第5课：元元分析？ (不是错字)。 谁知道。 也许有一天。

Conclusion of case 2: tidy everything! Even non-atomic objects.

案例2的结论 ：整理一切！ 甚至非原子物体。

最后一条建议：至少在您成为自己之前，征求专家的意见。 (One last piece of advice: Get an expert’s opinion, at least until you become one yourself.)

‘If only I had known that before. That could have saved me so much time and effort…’

如果我以前才知道这一点。 那可以节省我很多时间和精力……”

To the expert, your current challenges are yesterday’s resolution. They had already figured that out when we were still in kindergarten. They have spent their entire career just on that. Shoot them an email, ask a very clear question, with no dependencies proof-of-concepts examples, or case studies to demonstrate your challenge. My experience is that they would be happy to assist if you respect their time and authority.

对专家来说，您当前面临的挑战是昨天的解决方案。 他们在我们还在幼儿园的时候就已经知道了。 他们将整个职业生涯都用于此。 向他们发送电子邮件，提出一个非常明确的问题，没有依赖关系的概念证明示例，也没有案例研究来证明您的挑战。 我的经验是，如果您尊重他们的时间和权威，他们将很乐意为您提供帮助。

最后的话 (Final words)

When you figure out what type of tool/solution you are passionate about, make it happen! Don’t fool yourself with excuses why it is not a good time for your new tool to be created. Just do it!

当您确定您对哪种类型的工具/解决方案充满热情时，就去实现吧！ 不要以任何理由欺骗自己，为什么现在不是创建新工具的好时机。 去做就对了！

Don’t give up. Focus. Decide what you want to achieve. Do not be afraid to extend your scope, but do it with simple solutions! Refactor. It will be worth your time. Maybe not immediately, but in days to come. Be creative!

不要放弃 焦点。 确定要实现的目标。 不要害怕扩大您的范围，但是可以使用简单的解决方案来做到！ 重构。 这将是值得您度过的。 也许不是立即，而是未来的日子。 有创造力！

And last but not least, don’t be shy. Tell everyone about it. Share it with your community. Make the universe a better place with your solution. You may even earn an extra buck on the side. Who knows?

最后但同样重要的是，不要害羞。 告诉大家。 与您的社区分享。 用您的解决方案使宇宙成为更好的地方。 您甚至可以从侧面赚到额外的钱。 谁知道？

p.s.

ps

This post is dedicated with love to all of my former anxious collaborators / clients / bosses. I appreciate your patience, and wish I would have known the above before. You were there to assist and support me learning these lessons the hard way, for both good and for bad. Let me make it up to you. Shoot me an email and I will redo my old work in just a few lines of code, reflecting my current level of scope.

这篇帖子献给了我以前所有焦虑的合作者/客户/老板。 感谢您的耐心配合，并希望我早已知道以上内容。 您在那里是为了帮助和支持我以辛苦的方式学习这些课程，无论是好是坏。 让我来弥补你。 给我发一封电子邮件，然后我会用几行代码来重做我的旧作品，以反映我当前的范围。

Check more related topics here: https://drorberel.github.io/

在此处查看更多相关主题： https : //drorberel.github.io/

顾问：目前正在接受新项目！ (Consultant: currently accepting new projects!)

Useful reference:

有用的参考：

Clean Coder BlogOn the Diminished Capacity to Discuss Things Rationallyblog.cleancoder.comScope Creep in Project Management: Definition, Causes & SolutionsWhen a project stretches far beyond its original vision, it is called "scope creep". Scope creep in project management…www.workamajig.com

Clean Coder博客， 有关合理讨论问题的能力已减弱blog.cleancoder.com 项目管理中的范围蠕变：定义，原因和解决方案 当一个项目超出其最初的构想时，则称为“范围蠕变”。 项目管理的范围不断扩大…… www.workamajig.com

翻译自: https://www.freecodecamp.org/news/scope-creep-and-other-software-design-lessons-learned-the-hard-way-edacf021965b/

蠕变断裂 ansys