基于python渗透测试

by Shashi Kumar Raja

由Shashi Kumar Raja

Python中基于属性的测试简介 (Intro to property-based testing in Python)

In this article we will learn a unique and effective approach to testing called property-based testing. We will use Python , pytest and Hypothesis to implement this testing approach.

在本文中，我们将学习一种独特且有效的测试方法，称为基于属性的测试。 我们将使用Python ， pytest和Hypothesis来实现这种测试方法。

The article is going to use basic pytest concepts to explain property-based testing. I recommend that you read this article to quickly brush up your pytest knowledge.

本文将使用基本的pytest概念来解释基于属性的测试。 我建议您阅读本文以快速掌握pytest知识。

We will start with the conventional unit/functional testing method known as example-based testing — which most of us use. We try to find its shortcomings, and then move to the property-based approach to remove those shortcomings.

我们将从称为示例测试的常规单元/功能测试方法开始- 我们大多数人使用的。 我们尝试找到其缺点，然后转向基于属性的方法来消除这些缺点。

Every great magic trick consists of three parts or acts. The first part is called “The Pledge”. The magician shows you something ordinary: a deck of cards, a bird or a man. He shows you this object. Perhaps he asks you to inspect it to see if it is indeed real, unaltered, normal. But of course…it probably isn’t.

每个伟大的魔术都包括三个部分或动作。 第一部分称为“承诺”。 魔术师向您展示一些普通的东西 ：一副纸牌，一只鸟或一个人。 他向您展示了此对象。 也许他要求您检查它是否确实是真实的，未更改的，正常的。 但是，当然……可能不是。

第1部分：基于示例的测试 (Part 1: Example-based testing)

The approach of example-based testing has the following steps:

基于示例的测试方法具有以下步骤：

• given a test input I

  给定测试输入I

• when passed to function under test传递给被测函数时

• should return an output O

  应该返回输出O

So, basically we give a fixed input and expect a fixed output.

因此，基本上，我们给出固定的输入并期望得到固定的输出。

To understand this concept in layman’s terms:

要以外行的术语理解这个概念：

Assume we have a machine which takes plastic of any shape and any colour as input and produces a perfectly round plastic ball of the same colour as output.

假设我们有一台机器，将任何形状和任何颜色的塑料作为输入，并产生与输出颜色相同的完美圆形塑料球。

Now, to test this machine using example-based testing, we will follow below approach:

现在，要使用基于示例的测试来测试此机器，我们将采用以下方法：

1. take a blue-coloured raw plastic (fixed test data)

   采取蓝色的原始塑料( 固定测试数据 )

2. feed the plastic to machine将塑料喂入机器

3. expect a blue-coloured plastic ball as output(fixed test output)

   期望有一个蓝色的塑料球作为输出( 固定测试输出 )

Let’s see the same approach in a programmatic way.

让我们以编程方式查看相同的方法。

Prerequisite: make sure you have Python (ver 2.7 or above) and pytest installed.

先决条件：确保已安装Python (版本2.7或更高版本)和pytest 。

Create a directory structure like this:

创建这样的目录结构：

- demo_tests/    - test_example.py

We will write one small function sum inside file test_example.py . This accepts two numbers — num1 and num2 — as parameters and returns the addition of both numbers as result.

我们将在文件test_example.py编写一个小函数sum 。 它接受两个数字num1和num2作为参数，并返回两个数字的和作为结果。

def sum(num1, num2):    """It returns sum of two numbers"""    return num1 + num2

Now, lets write a test to test this sum function following the conventional method.

现在，让我们编写一个测试以按照常规方法测试此求和函数。

import pytest

#make sure to start function name with testdef test_sum():    assert sum(1, 2) == 3

Here you can see that, we are passing the two values 1 and 2 and expecting the sum to return 3.

在这里您可以看到，我们传递了两个值1和2并期望总和返回3 。

Run the tests by traversing to demo_tests folder and then running following command:

通过遍历demo_tests文件夹，然后运行以下命令来运行测试：

pytest test_example.py -v

Is this test enough to verify the functionality of the sum function?

该测试足以验证sum函数的功能吗？

You might be thinking, of course not. We will write more tests using the pytest parametrize feature which will execute this test_sum function for all the given values.

您可能在想，当然不是。 我们将使用pytest parametrize功能编写更多测试，该功能将对所有给定值执行此test_sum函数。

import pytest

@pytest.mark.parametrize('num1, num2, expected',[(3,5,8),              (-2,-2,-4), (-1,5,4), (3,-5,-2), (0,5,5)])def test_sum(num1, num2, expected):        assert sum(num1, num2) == expected

Using five tests has given more confidence about the functionality. All of them passing feels like bliss.

使用五个测试使人们对该功能有了更多的信心。 他们所有人过去的感觉就像幸福。

But, if you look more closely we are doing the same thing we did above but for more number of values. We are still not covering several of the edge cases.

但是 ，如果您仔细观察的话，我们正在做的是上面所做的相同的事情，只是更多的值。 我们仍然没有涵盖几个极端情况。

So, we have discovered the first pain point with this method of testing:

因此，我们发现了这种测试方法的第一个痛点：

问题1：测试的详尽程度取决于编写测试的人 (Issue 1: Test exhaustiveness depends on the person writing the test)

They may choose to write 5 or 50 or 500 test cases but still remain unsure whether they have safely covered most, if not all, the edge cases.

他们可以选择编写5个，50个或500个测试用例，但仍然不确定他们是否已经安全地涵盖了大多数(如果不是全部)边缘案例。

This brings us to our second pain point:

这将我们带到第二个痛点：

问题2 –由于对需求的了解不明确/模棱两可而导致的非稳健测试 (Issue 2 — Non-robust tests due to unclear/ambiguous requirement understanding)

When we were told to write our sum function, what specific details were conveyed?

当要求我们编写sum函数时，传达了哪些具体细节？

Were we told:

我们被告知：

• what kind of input our function should expect?我们的功能应该期望什么样的输入？
• how our function should behave in unexpected input scenarios?在意外的输入场景中，我们的函数应该如何表现？
• what kind of output our function should return?我们的函数应该返回什么样的输出？

To be more precise, if you consider the sum function we have written above:

更精确地说，如果考虑上面我们写的sum函数：

• do we know if num1, num2 should be an int or float? Can they also be sent as type string or any other data type?

  我们是否知道num1 ， num2应该是int还是float ？ 它们也可以作为string类型或任何其他数据类型发送吗？

• what is the minimum and maximum value of num1 and num2 that we should support?

  我们应该支持的num1和num2的最小值和最大值是多少？

• how should the function behave if we get null inputs?

  如果我们得到null输入，该函数应该如何表现？

• should the output returned by the sum function be int or float or string or any other data type?

  sum函数返回的输出应该是int还是float或string或任何其他数据类型？

• in what scenarios should it display error messages?在什么情况下应该显示错误消息？

Also, the worst case scenario of the above test case writing approach is that these test cases can be fooled to pass by buggy functions.

同样，上述测试用例编写方法的最坏情况是，这些测试用例可能被错误的功能欺骗 。

Let’s re-write our sum function in a way that errors are introduced but the tests which we have written so far still passes.

让我们以引入错误的方式重新编写我们的sum函数，但是到目前为止我们编写的测试仍然可以通过。

def sum(num1, num2):    """Buggy logic"""       if num1 == 3 and num2 == 5:        return 8    elif num1 == -2 and num2  == -2 :        return -4    elif num1 == -1 and num2 == 5 :        return 4    elif num1 == 3 and num2 == -5:        return -2    elif num1 == 0 and num2 == 5:        return 5

Now let’s dive into property-based testing to see how these pain points are mitigated there.

现在，让我们深入进行基于属性的测试，以了解如何缓解这些痛苦点。

The second act is called “The Turn”. The magician takes the ordinary something and makes it do something extraordinary. Now you’re looking for the secret… but you won’t find it, because of course you’re not really looking. You don’t really want to know. You want to be fooled.

第二幕称为“转弯”。 魔术师把平凡的东西拿来做，使它变得非凡。 现在，您正在寻找秘密……但是您找不到它，因为您当然不是真正的寻找者。 你真的不想知道。 你想上当。

第2部分：基于属性的测试 (Part 2: Property-based testing)

简介和测试数据生成 (Intro and test data generation)

Property-based testing was first introduced by the QuickCheck framework in Haskell. As per fast-check’s documentation, which is another property based testing library-

基于属性的测试首先由Haskell中的QuickCheck框架引入。 根据快速检查的文档，这是另一个基于属性的测试库，

Property based testing frameworks check the truthfulness of properties. A property is a statement like:

基于属性的测试框架检查属性的真实性。 属性是如下语句：

for all (x, y, …)

全部(x，y，…)

such as precondition(x, y, …) holds

如前提(x，y，…)成立

property(x, y, …) is true.

property(x，y，…)为true 。

To understand this let’s go back to our plastic ball generating machine example.

为了理解这一点，让我们回到我们的塑料球发生机示例。

The property based testing approach of that machine will be:

该机器的基于属性的测试方法将是：

1. take a huge selection of plastics as input (all(x, y, …))

   大量选择塑料作为输入( all(x, y, …) )

2. make sure all of them are colored (precondition(x, y, …))

   确保它们全部都是彩色的( precondition(x, y, …) )

3. the output satisfies following property (property(x, y, …)) -

   输出满足以下属性( property(x, y, …) )-

• output is round/spherical in shape

  输出为圆形/球形

• output is colored

  输出是彩色的

• color of the output is one of the colors present in color band

  输出的颜色是色带中存在的颜色之一

Notice how from fixed values of input and output we have generalized our test data and output in such a way that the property should hold true for all the valid inputs. This is property-based testing.

请注意，如何从输入和输出的固定值对测试数据和输出进行一般化 ，以使该属性对于所有有效输入都适用。 这是基于属性的测试。

Also, notice that when thinking in terms of properties we have to think harder and in a different way. Like when we came up with the idea that since our output is a ball it should be round in shape, another question will strike you - whether the ball should be hollow or solid?

另外，请注意，在考虑属性时，我们必须加倍努力，以不同的方式思考。 就像当我们提出这样一个想法时，既然我们的输出是一个球，它应该是圆形的，那么另一个问题就会出现在您的面前- 球应该是空心的还是实心的 ？

So, by making us think harder and question more about the requirement, the property-based testing approach is making our implementation of the requirement robust.

因此，通过使我们更加认真思考和对需求提出更多疑问，基于属性的测试方法使我们对需求的实施变得更加可靠。

Now, let’s return to our sum function and test it by using the property-based approach.

现在，让我们回到sum函数并使用基于属性的方法对其进行测试。

The first question which arises here is: what should be the input of the sum function?

这里出现的第一个问题是： sum函数的输入应该是什么？

For the scope of this article we will assume that any pair of integers from the integer set is a valid input.

对于本文的范围，我们将假定整数集中的任何一对整数都是有效输入。

So, any set of integer values lying in the above coordinate system will be a valid input to our function.

因此，位于上述坐标系中的任何一组整数值将是我们函数的有效输入。

The next question is: how to get such input data?

下一个问题是：如何获取此类输入数据？

The answer to this is: a property-based testing library provides you the feature to generate huge set of desired input data following a precondition.

答案是：基于属性的测试库为您提供了根据前提条件生成大量所需输入数据的功能。

In Python, Hypothesis is a property-testing library which allows you to write tests along with pytest. We are going to make use of this library.

在Python中， 假设是一个属性测试库，可让您与pytest一起编写测试。 我们将利用这个库。

The entire documentation of Hypothesis is beautifully written and available ➡️ here and I recommend you to go through it.

假设的完整文档编写精美，可在此处找到➡️ 我建议您仔细阅读。

To install Hypothesis:

要安装假设：

pip install hypothesis

and we are good to use hypothesis with pytest.

而且我们很高兴将假设与pytest结合使用。

Now, let’s rewrite test_sum function — which we wrote earlier — with new data sets generated by Hypothesis.

现在，让我们用由假设生成的新数据集重写我们先前编写的test_sum函数。

from hypothesis import given

import hypothesis.strategies as st

import pytest

@given(st.integers(), st.integers())def test_sum(num1, num2):    assert sum(num1, num2) == num1 + num2

• The first line simply imports given from Hypothesis. The @given decorator takes our test function and turns it into a parametrized one. When called, this will run the test function over a wide range of matching data. This is the main entry point to Hypothesis.

  第一行简单的进口given的假说。 @given装饰器接受我们的测试功能，并将其转换为参数化的功能。 调用时，它将在广泛的匹配数据上运行测试功能。 这是假设的主要切入点。

• The second line imports strategies from Hypothesis. strategies provides the feature to generate test data. Hypothesis provides strategies for most built-in types with arguments to constrain or adjust the output. As well, higher-order strategies can be composed to generate more complex types.

  第二行从假设中导入strategies 。 策略提供了生成测试数据的功能 。 假设为大多数内置类型提供了带有约束或调整输出的参数的策略。 同样，可以组合更高阶的策略以生成更复杂的类型。

• You can generate any or mix of the following things using strategies:您可以使用策略生成以下任何东西或混合使用：

'nothing','just', 'one_of','none','choices', 'streaming','booleans', 'integers', 'floats', 'complex_numbers', 'fractions','decimals','characters', 'text', 'from_regex', 'binary', 'uuids','tuples', 'lists', 'sets', 'frozensets', 'iterables','dictionaries', 'fixed_dictionaries','sampled_from', 'permutations','datetimes', 'dates', 'times', 'timedeltas','builds','randoms', 'random_module','recursive', 'composite','shared', 'runner', 'data','deferred','from_type', 'register_type_strategy', 'emails'

• Here we have generated integers()set using strategies and passed it to @given.

  在这里，我们使用策略生成了integers()集合，并将其传递给@given 。

• So, our test_sum function should run for all the iterations of given input.

  因此，我们的test_sum函数应针对给定输入的所有迭代运行。

Let’s run it and see the result.

让我们运行它并查看结果。

You might be thinking, I can’t see any difference here. What’s so special about this run?

您可能在想，我看不出有什么区别。 这次跑步有何特别之处？

Well, to see the magical difference, we need to run our test by setting the verbose option. Don’t confuse this verbose with the -v option of pytest.

好吧，要查看神奇的区别，我们需要通过设置verbose选项来运行测试。 不要将此冗长的内容与pytest的-v选项混淆。

from hypothesis import given, settings, Verbosity

import hypothesis.strategies as stimport pytest

@settings(verbosity=Verbosity.verbose)@given(st.integers(), st.integers())def test_sum(num1, num2):    assert sum(num1, num2) == num1 + num2

settings allows us to tweak the default test behavior of Hypothesis.

settings允许我们调整假设的默认测试行为。

Now let’s re-run the tests. Also include -s this time to capture the stream output in pytest.

现在，让我们重新运行测试。 这次还包括-s以捕获pytest中的流输出。

pytest test_example.py -v -s

Look at the sheer number of test-cases generated and run. You can find all sorts of cases here, such as 0, large numbers, and negative numbers.

查看生成并运行的大量测试用例。 您可以在这里找到各种情况，例如0，大数和负数。

You might be thinking, it’s impressive, but I can’t find my favorite test case pair (1,2 ) here. What if I want that to run?

您可能会想，这令人印象深刻，但是我在这里找不到我最喜欢的测试用例对(1,2) 。 如果我要运行该怎么办？

Well, fear not, Hypothesis allows you to run a given set of test cases every time if you want by using the @example decorator.

好吧，不用担心，假设可以让您每次使用@ example装饰器运行给定的测试用例集。

from hypothesis import given, settings, Verbosity, example

import hypothesis.strategies as stimport pytest

@settings(verbosity=Verbosity.verbose)@given(st.integers(), st.integers())@example(1, 2)def test_sum(num1, num2):    assert sum(num1, num2) == num1 + num2

Also, notice that each run will always generate a new jumbled up test case following the test generation strategy, thus randomizing the test run.

另外，请注意，每次运行将始终遵循测试生成策略来生成新的混乱测试用例，从而使测试运行随机化。

So, this solves our first pain point- the exhaustiveness of test cases.

因此，这解决了我们的第一个痛点-测试用例的详尽性。

努力思考要测试的属性 (Thinking hard to come up with properties to test)

So far, we saw one magic of property-based testing which generates desired test data on the fly.

到目前为止，我们看到了基于属性的测试的一种魔力，它可以动态生成所需的测试数据。

Now let’s come to the part where we need to think hard and in a different way to create such tests which are valid for all test inputs but unique to sum function.

现在让我们进入需要认真思考的部分，以另一种方式来创建对所有测试输入均有效但 sum函数唯一的测试。

1 + 0 = 10 + 1 = 15 + 0 = 5-3 + 0 = -38.5 + 0 = 8.5

Well, that’s interesting. It seems like adding 0 to a number results in the same number as sum. This is called the identity property of addition.

好吧，那很有趣。 似乎在数字上加上0会得到与sum相同的数字。 这称为加法的标识属性。

Let’s see one more:

让我们再看一个：

2 + 3 = 53 + 2 = 5

5 + (-2) = 3-2 + 5 = 3

It looks like we found one more unique property. In addition the order of parameters doesn’t matter. Placed left or right of the + sign they give the same result. This is called the commutative property of addition.

看来我们找到了另一个独特的属性。 另外，参数的顺序无关紧要。 在+号的左侧或右侧放置它们可得到相同的结果。 这称为加法的交换性质。

There is one more, but I want you to come up with it.

还有一个，但我希望您提出。

Now, we will re-write our test_sum to test these properties:

现在，我们将重新编写test_sum以测试以下属性：

from hypothesis import given, settings, Verbosity

import hypothesis.strategies as stimport pytest

@settings(verbosity=Verbosity.verbose)@given(st.integers(), st.integers())def test_sum(num1, num2):    assert sum(num1, num2) == num1 + num2

# Test Identity property    assert sum(num1, 0) = num1     #Test Commutative property      assert sum(num1, num2) == sum(num2, num1)

Our test is now exhaustive — we have also converted the tests to make them more robust. Thus, we solved our second pain point: non-robust test cases.

现在，我们的测试是详尽无遗的-我们还对测试进行了转换，以使其更加可靠。 因此，我们解决了第二个痛点： 非健壮的测试用例 。

Just for curiosity’s sake, let’s try to fool this test with that buggy code we used before.

只是出于好奇，让我们尝试用之前使用的错误代码来欺骗该测试。

As an old proverb says- fool me once, shame on you, fool me twice, shame on me.

就像一句古老的谚语所说：愚弄我一次，羞辱你，愚弄我两次，羞辱我。

You can see that it caught an error. Falsifying example: test_sum(num1=0, num2=0). It simply means that our expected property didn't hold true for these pairs of test cases, thus the failure.

您可以看到它捕获了一个错误。 Falsifying example: test_sum(num1=0, num2=0). 这仅表示我们的预期属性不适用于这对测试用例，从而导致失败。

But you wouldn’t clap yet. Because making something disappear isn’t enough; you have to bring it back. That’s why every magic trick has a third act, the hardest part, the part we call “The Prestige”.

但是您还不会鼓掌。 因为使某事消失是不够的。 你必须把它带回来。 这就是为什么每个魔术都有第三幕，最难的部分，我们称之为“威望”的部分。

第3部分：收缩失败 (Part 3: Shrinking failures)

Shrinking is the process by which Hypothesis tries to produce human-readable examples when it finds a failure. It takes a complex example and turns it into a simpler one.

缩小是假设在发现故障时试图产生易于理解的示例的过程。 它采用了一个复杂的示例，并将其变成一个简单的示例。

To demonstrate this feature, let’s add one more property to our test_sum function which says num1 should be less than or equal to 30.

为了演示此功能，让我们在test_sum函数中再添加一个属性，该属性表示num1应该小于或等于30.

from hypothesis import given, settings, Verbosity

import hypothesis.strategies as stimport pytest

@settings(verbosity=Verbosity.verbose)@given(st.integers(), st.integers())def test_sum(num1, num2):    assert sum(num1, num2) == num1 + num2

# Test Identity property    assert sum(num1, 0) = num1     #Test Commutative property      assert sum(num1, num2) == sum(num2, num1)    assert num1 <= 30

After running this test, you will get an interesting output log on the terminal here:

运行此测试后，您将在此处的终端上获得有趣的输出日志：

collected 1 item

test_example.py::test_sum Trying example: test_sum(num1=0, num2=-1)Trying example: test_sum(num1=0, num2=-1)Trying example: test_sum(num1=0, num2=-29696)Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=-1763, num2=47)Trying example: test_sum(num1=6, num2=1561)Trying example: test_sum(num1=-24900, num2=-29635)Trying example: test_sum(num1=-13783, num2=-20393)

#Till now all test cases passed but the next one will fail

Trying example: test_sum(num1=20251, num2=-10886)assert num1 <= 30AssertionError: assert 20251 <= 30

#Now the shrinking feature kicks in and it will try to find the simplest value for which the test still fails

Trying example: test_sum(num1=0, num2=-2)Trying example: test_sum(num1=0, num2=-1022)Trying example: test_sum(num1=-165, num2=-29724)Trying example: test_sum(num1=-14373, num2=-29724)Trying example: test_sum(num1=-8421504, num2=-8421376)Trying example: test_sum(num1=155, num2=-10886)assert num1 <= 30AssertionError: assert 155 <= 30

# So far it has narrowed it down to 155

Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=64, num2=0)assert num1 <= 30AssertionError: assert 64 <= 30

# Down to 64

Trying example: test_sum(num1=-30, num2=0)Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=31, num2=0)

# Down to 31

Trying example: test_sum(num1=-30, num2=0)Falsifying example: test_sum(num1=31, num2=0)FAILED

# And it finally concludes (num1=31, num2=0) is the simplest test data for which our property doesn't hold true.

One more good feature — its going to remember this failure for this test and will include this particular test case set in the future runs to make sure that the same regression doesn’t creep in.

另一个好功能- 它会记住此测试的失败 ，并将在将来运行时包含此特定的测试用例集，以确保不会出现相同的回归。

This was a gentle introduction to the magic of property based testing. I recommend all of you try this approach in your day to day testing. Almost all major programming languages have property based testing support.

这是对基于属性的测试的神奇介绍。 我建议大家在日常测试中尝试这种方法。 几乎所有主要的编程语言都具有基于属性的测试支持。

You can find the entire code used here in my ? github repo.

您可以在我的？中找到此处使用的完整代码。 g ithub回购。

If you liked the content show some ❤️

如果您喜欢内容，请显示一些❤️

翻译自: https://www.freecodecamp.org/news/intro-to-property-based-testing-in-python-6321e0c2f8b/

基于python渗透测试