用数字暗示我喜欢你

Time has come for a deeper dive into the number crunching that is also a part of being in quality assurance. I like numbers. I love spreadsheets and nothing beats Excel in the world of numbers. Numbers are themselves irrelevant, but the story behind any set of numbers is fascinating, fantastic and sometimes also a horror. I will dig a bit into the numbers we use in Unity QA, how we get them and what we use them for.

现在是时候更深入地研究数字运算了，这也是保证质量的一部分。 我喜欢数字。 我喜欢电子表格，在数字世界中没有什么能比Excel更好。 数字本身是无关紧要的，但是任何一组数字背后的故事都是令人着迷的，奇妙的，有时甚至是恐怖的。 我将深入研究我们在Unity QA中使用的数字，如何获得它们以及将它们用于什么。

There’s one specific term I use throughout this blog post, which is an Incident. Whenever a user submits a bug report through the bug reporter in Unity, we call it an Incident. QA processes the Incidents by reproducing it and once we have a reproduction we convert it to a bug. This is important in the following.

我在整个博客文章中使用了一个特定的术语，即“事件”。 每当用户通过Unity中的错误报告程序提交错误报告时，我们都将其称为事件。 质量保证通过复制事件来处理事件，一旦我们复制了事件，便将其转换为错误。 在下面这很重要。

在黑暗中导航 (Navigating in the dark)

Developing software is a journey in unknown territory, especially when you are trying to invent new cool stuff which has not been done before, so you need to figure out how to tell whether you’re on the right path or not. Wandering aimlessly will get you somewhere, but if you have a target and a few tools, you can get to a specific place instead of accepting whatever you stumble upon.

开发软件是一段未知领域的旅程，尤其是当您尝试发明以前从未做过的新奇事物时，因此您需要弄清楚如何判断自己是否走对了道路。 漫无目的地游荡会带您到某个地方，但是如果您有目标和一些工具，则可以到达特定的位置，而不用接受偶然发现的一切。

Ask the right questions and numbers can help you find a way. Here’s some of the questions I have asked:

提出正确的问题和数字可以帮助您找到方法。 这是我问的一些问题：

These are just some of the questions I asked myself.

这些只是我问自己的一些问题。

聚会 (The Gathering)

The hardest part of tracking numbers is to make sure the data quality of these numbers is good enough. Lots of issues arise once you try to dig into a source of data and our journey has been the same as usual. A very simple thing is to change the type of a Fogbugz case from a “Bug” to an “Incident” when you guys report it to us. This change enables us to setup a better process for QA to reproduce the incidents and turn them into bugs once we have a successful repro. This gives devs the comfort that a bug has actually been reproduced inside Unity and is not related to anything specific on the user computer and, equally important; we can now start to have confidence in our bug count.

跟踪号码最困难的部分是确保这些号码的数据质量足够好。 一旦您尝试挖掘数据源，就会遇到很多问题，而我们的旅程与往常一样。 一个非常简单的事情是，当你们向我们报告时，将Fogbugz案件的类型从“ Bug”更改为“ Incident”。 这一更改使我们能够为质量检查设置更好的流程，以重现事件，并在成功复制后将其转变为错误。 这使开发人员可以放心，错误实际上是在Unity内部重现的，并且与用户计算机上的特定内容无关，并且同样重要。 现在我们可以开始对我们的错误计数有信心了。

This was just one example of how a small change in process can help you attain more certainty about your numbers. I will show some of this data further down in this post.

这只是流程中的微小变化如何帮助您获得数字确定性的一个示例。 在这篇文章中，我将进一步展示一些数据。

A more techie part of gathering data is to pull in data from different sources, process them, link them up and build a cube (a pre-processed database with a special format for very fast reporting) for analysis on it. Currently we grab data from Fogbugz and Google Analytics, where we can track all sorts of bugs statistics and correlate them with the analytics data on a date and version level.

收集数据的一个比较技术性的部分是从不同来源提取数据，对其进行处理，将它们链接起来并构建一个多维数据集(具有特殊格式的预处理数据库，可以非常快速地生成报告)以对其进行分析。 目前，我们从Fogbugz和Google Analytics(分析)中获取数据，我们可以在其中跟踪各种错误统计信息，并将它们与日期和版本级别的分析数据相关联。

The technical part of this is done on SQL Server with an analysis services cube and reporting services for standard reports and Excel for custom analysis. The details of this setup are beyond this post, but you can find plenty of information on this setup from other sites.

技术部分是在SQL Server上完成的，其中包括用于标准报告的Analysis Services多维数据集和报告服务，以及用于自定义分析的Excel。 此设置的详细信息超出了本文的范围，但是您可以从其他站点上找到有关此设置的大量信息。

展示给我看 (Show it to me)

每日收养 (Daily Adoption)

This chart shows the action we received on 4.0 beta  2 and 4.0 beta 3. The lines represent the number of times the editor was opened in each version and the bars represent the number of incidents we received from the beta testers, all on a day by day basis. We can use this to compare how the adoption is of a specific version is going and unfortunately these numbers we see in this chart are not what we had hoped for, which leads us to search for explanations to why our beta testers are not as active now as they were during 3.5; we must be doing something the wrong way for this to happen. The effect of having a smaller adoption and less feedback from our community is extremely real in the sense that we might have to prolong a release period if we don’t feel we have had enough testing done on the product.

此图表显示了我们在4.0 beta 2和4.0 beta 3上收到的操作。线条表示在每个版本中打开编辑器的次数，而横线表示一天之内从Beta测试人员收到的事件数量。日基础。 我们可以使用它来比较采用特定版本的情况，不幸的是，我们在此图表中看到的这些数字并非我们所希望的，这导致我们寻找为什么我们的beta测试人员现在不那么活跃的解释就像在3.5时一样； 我们必须以错误的方式做这件事。 如果我们认为我们没有对产品进行足够的测试，则可能必须延长发布时间，这是因为采用更少的产品和减少来自社区的反馈是非常真实的。

版本采用 (Version Adoption)

With this report we can see the overall status of how often an editor has been started on a specific version. We focus a lot on how to increase the number of people using these beta versions, so when we saw a very, very limited amount of feedback and adoption from our alpha group, we pushed hard to get a beta out to  the larger beta group to increase the test surface. Likewise we are very eager to make a public version because the first public beta of 3.5 has been opened more than 1 million times (it is still being opened 10.000 times every week!). The Good-Bad Bug Ratio tells us how many of the bugs we have converted which ended up being fixed vs. those that end up being By Design or Won’t Fix, a number we track to find really great guys in our beta testing program and sometimes hand out a reward to them.

通过此报告，我们可以查看在特定版本上启动编辑器的频率的总体状态。 我们非常关注如何增加使用这些Beta版本的人数，因此，当我们看到来自alpha组的反馈非常有限时，我们努力将Beta推广到更大的beta组，以增加测试表面。 同样，我们非常渴望发布一个公开版本，因为3.5的第一个公开测试版已打开超过100万次(它仍每周打开10.000次！)。 良好的错误率告诉我们，已经转换的错误中有多少是最终被修复的，而最终是由设计或无法修复的，我们跟踪的数量可以在我们的Beta测试计划中找到真正的好人有时会给他们奖励

One outcome of monitoring these numbers was that we realized something was wrong with beta three and after a bit of thinking it turned out we had forgotten to set the auto update script to go to the next beta version, so many were staying on 4.0 beta 1 instead of moving to beta 3.

监控这些数字的结果是，我们意识到Beta 3出了点问题，经过一番思考后，我们忘记了将自动更新脚本设置为下一个Beta版本，所以很多人都停留在4.0 beta 1上。而不是转到beta 3。

事故状态 (Incidents Status)

This overview tells us how we’re doing on handling incidents coming from the bugreporter in Unity. Every time someone submits a bug report, we treat it as an incident which has to be reproduced by QA before we convert it to a bug and this overview can tell us if we are behind on the queues, how many were converted and how good the converted bugs were.

此概述告诉我们如何处理来自Unity中bug报告程序的事件。 每次有人提交错误报告时，我们都将其视为必须由质量检查重现的事件，然后再将其转换为错误，此概述可以告诉我们是否排在队列中，转换了多少，以及转换的错误。

I monitor this regularly to make sure we get all of the beta feedback processed. It is the most important information we get from you guys out there, so we have to handle it.

我会定期进行监控，以确保我们处理了所有的Beta反馈。 这是我们从你们那里获得的最重要的信息，因此我们必须加以处理。

每个区域的错误 (Bugs per Area)

This is the bug situation we have in 4.0 right now, stacked by priority. All these bugs are destined for resolution before we ship, although reality sometimes catches up. With this chart we see where we have big chunks which need to be mitigated. Sometimes teams will cry for help from other teams to get their area under control. Fairly straight forward stuff here.

这是我们目前在4.0中存在的错误情况，按优先级排列。 所有这些错误注定要在我们发货之前加以解决，尽管有时有时会赶上现实。 通过此图表，我们可以看到需要缓解的地方。 有时，团队会哭泣寻求其他团队的帮助，以控制他们的区域。 这里的东西很简单。

操作系统 (Operating Systems)

We collect data about which operating systems are used for developing on Unity and sometimes the results are…interesting.

我们收集有关在Unity上使用哪些操作系统进行开发的数据，有时结果令人感兴趣。

Windows accounts for 80% of all users using the editor.

Windows占使用编辑器的所有用户的80％。

We are effectively no longer testing the editor on Windows Vista because the adoption of this platform is so low, a decision directly derived from the above statistics. Not to be confused with the webplayer, which is still tested on Windows Vista!

实际上，我们不再在Windows Vista上测试该编辑器，因为该平台的采用率非常低，这是直接根据上述统计数据得出的决定。 不要与webplayer混淆，该webplayer仍在Windows Vista上进行了测试！

During the last month users opened Unity3D on a Windows XP machine 414.221 times, amounting to about 14% of all Windows users, so we are still having this one in our test runs.

在过去的一个月中，用户在Windows XP计算机上打开Unity3D的次数为414.221次，约占所有Windows用户的14％，因此我们在测试运行中仍然使用此工具。

结语 (Wrapping it up)

I’ve tried to paint a picture of some of the things we do here in Unity. Often we need to navigate in the dark, but wherever possible we try to shed light on the way. I hope you found it interesting and if you would like to join us, go to http://www.unity3d.com/company/jobs .

我试图描绘一些我们在Unity中所做的事情。 通常，我们需要在黑暗中导航，但在可能的情况下，我们会尽力为您提供帮助。 希望您觉得它很有趣，如果您想加入我们，请访问http://www.unity3d.com/company/jobs 。

翻译自: https://blogs.unity3d.com/2012/08/01/i-like-numbers/

用数字暗示我喜欢你