python gil全局锁

The Python Global Interpreter Lock or GIL, in simple words, is a mutex (or a lock) that allows only one thread to hold the control of the Python interpreter.

简而言之，Python全局解释器锁或GIL是一种互斥锁（或锁），仅允许一个线程持有Python解释器的控制权。

This means that only one thread can be in a state of execution at any point in time. The impact of the GIL isn’t visible to developers who execute single-threaded programs, but it can be a performance bottleneck in CPU-bound and multi-threaded code.

这意味着在任何时间点只有一个线程可以处于执行状态。 对于执行单线程程序的开发人员而言，GIL的影响并不明显，但它可能是CPU绑定和多线程代码的性能瓶颈。

Since the GIL allows only one thread to execute at a time even in a multi-threaded architecture with more than one CPU core, the GIL has gained a reputation as an “infamous” feature of Python.

由于即使在具有多个CPU内核的多线程体系结构中，GIL一次一次只允许执行一个线程，因此GIL被誉为Python的“臭名昭著”功能。

In this article you’ll learn how the GIL affects the performance of your Python programs, and how you can mitigate the impact it might have on your code.

在本文中，您将学习GIL如何影响Python程序的性能，以及如何减轻GIL对代码的影响。

GIL为Python解决了什么问题？ (What problem did the GIL solve for Python?)

Python uses reference counting for memory management. It means that objects created in Python have a reference count variable that keeps track of the number of references that point to the object. When this count reaches zero, the memory occupied by the object is released.

Python使用引用计数进行内存管理。 这意味着用Python创建的对象具有引用计数变量，该变量跟踪指向该对象的引用数。 当此计数达到零时，将释放对象占用的内存。

Let’s take a look at a brief code example to demonstrate how reference counting works:

让我们看一个简短的代码示例，以演示引用计数的工作原理：

 >>> >>>  import import sys
sys
>>> >>>  a a = = []
[]
>>> >>>  b b = = a
a
>>> >>>  syssys .. getrefcountgetrefcount (( aa )
)
3
3

In the above example, the reference count for the empty list object [] was 3. The list object was referenced by a, b and the argument passed to sys.getrefcount().

在上面的示例中，空列表对象[]的引用计数为3。列表对象由a ， b引用，并且参数传递给sys.getrefcount() 。

Back to the GIL:

回到GIL：

The problem was that this reference count variable needed protection from race conditions where two threads increase or decrease its value simultaneously. If this happens, it can cause either leaked memory that is never released or, even worse, incorrectly release the memory while a reference to that object still exists. This can can cause crashes or other “weird” bugs in your Python programs.

问题在于该引用计数变量需要保护，以防止两个线程同时增加或减少其值的竞争条件。 如果发生这种情况，则可能导致从未释放的内存泄漏，或者更糟糕的是，在仍然存在对该对象的引用的情况下，错误地释放了内存。 这可能会导致崩溃或Python程序中的其他“怪异”错误。

This reference count variable can be kept safe by adding locks to all data structures that are shared across threads so that they are not modified inconsistently.

通过将锁添加到所有在线程之间共享的数据结构中，以便不被不一致地修改，可以保持此引用计数变量的安全。

But adding a lock to each object or groups of objects means multiple locks will exist which can cause another problem—Deadlocks (deadlocks can only happen if there is more than one lock). Another side effect would be decreased performance caused by the repeated acquisition and release of locks.

但是，将锁添加到每个对象或一组对象意味着存在多个锁，这可能会导致另一个问题-死锁（只有在有多个锁的情况下才会发生死锁）。 另一个副作用是由于重复获取和释放锁而导致性能降低。

The GIL is a single lock on the interpreter itself which adds a rule that execution of any Python bytecode requires acquiring the interpreter lock. This prevents deadlocks (as there is only one lock) and doesn’t introduce much performance overhead. But it effectively makes any CPU-bound Python program single-threaded.

GIL是解释器本身的单个锁，它添加了一个规则，即任何Python字节码的执行都需要获取解释器锁。 这样可以防止死锁（因为只有一个锁）并且不会带来太多的性能开销。 但这实际上使所有受CPU约束的Python程序都是单线程的。

The GIL, although used by interpreters for other languages like Ruby, is not the only solution to this problem. Some languages avoid the requirement of a GIL for thread-safe memory management by using approaches other than reference counting, such as garbage collection.

尽管解释器用于其他语言（例如Ruby），但GIL并不是解决此问题的唯一方法。 某些语言通过使用引用计数以外的方法（例如垃圾回收）来避免对线程安全的内存管理使用GIL的要求。

On the other hand, this means that those languages often have to compensate for the loss of single threaded performance benefits of a GIL by adding other performance boosting features like JIT compilers.

另一方面，这意味着这些语言通常必须通过添加其他性能提升功能（如JIT编译器）来弥补GIL的单线程性能优势的损失。

为什么选择GIL作为解决方案？ (Why was the GIL chosen as the solution?)

So, why was an approach that is seemingly so obstructing used in Python? Was it a bad decision by the developers of Python?

那么，为什么在Python中使用了一种看起来如此阻碍的方法呢？ Python开发人员是否会做出错误的决定？

Well, in the words of Larry Hastings, the design decision of the GIL is one of the things that made Python as popular as it is today.

好吧，用Larry Hastings的话来说， GIL的设计决定是使Python像今天一样流行的原因之一。

Python has been around since the days when operating systems did not have a concept of threads. Python was designed to be easy-to-use in order to make development quicker and more and more developers started using it.

自从操作系统没有线程概念以来，Python就已经存在了。 Python被设计为易于使用，以加快开发速度，越来越多的开发人员开始使用它。

A lot of extensions were being written for the existing C libraries whose features were needed in Python. To prevent inconsistent changes, these C extensions required a thread-safe memory management which the GIL provided.

现有的C库正在编写许多扩展，这些C需要在Python中实现其功能。 为了防止不一致的更改，这些C扩展需要GIL提供的线程安全内存管理。

The GIL is simple to implement and was easily added to Python. It provides a performance increase to single-threaded programs as only one lock needs to be managed.

GIL易于实现，并且很容易添加到Python中。 由于只需要管理一个锁，因此它可以提高单线程程序的性能。

C libraries that were not thread-safe became easier to integrate. And these C extensions became one of the reasons why Python was readily adopted by different communities.

非线程安全的C库变得易于集成。 这些C扩展成为Python被不同社区轻易采用的原因之一。

As you can see, the GIL was a pragmatic solution to a difficult problem that the CPython developers faced early on in Python’s life.

如您所见，GIL是CPython开发人员在Python生命早期面临的一个难题的务实解决方案。

对多线程Python程序的影响 (The impact on multi-threaded Python programs)

When you look at a typical Python program—or any computer program for that matter—there’s a difference between those that are CPU-bound in their performance and those that are I/O-bound.

当您查看典型的Python程序（或与此相关的任何计算机程序）时，受CPU限制的性能与受I / O限制的性能之间是有区别的。

CPU-bound programs are those that are pushing the CPU to its limit. This includes programs that do mathematical computations like matrix multiplications, searching, image processing, etc.

受CPU约束的程序是将CPU推到极限的程序。 这包括进行数学计算的程序，例如矩阵乘法，搜索，图像处理等。

I/O-bound programs are the ones that spend time waiting for Input/Output which can come from a user, file, database, network, etc. I/O-bound programs sometimes have to wait for a significant amount of time till they get what they need from the source due to the fact that the source may need to do its own processing before the input/output is ready, for example, a user thinking about what to enter into an input prompt or a database query running in its own process.

受I / O约束的程序是花费时间等待输入/输出的程序，它可能来自用户，文件，数据库，网络等。受I / O约束的程序有时必须等待大量时间，直到它们进入由于源可能需要在输入/输出准备好之前进行自己的处理，因此可以从源那里获得他们需要的东西，例如，用户考虑要在输入提示中输入什么内容或在其输入中运行数据库查询自己的过程。

Let’s have a look at a simple CPU-bound program that performs a countdown:

让我们看一个执行倒计时的简单的受CPU约束的程序：

Running this code on my system with 4 cores gave the following output:

在具有4个内核的系统上运行此代码，输出如下：

 $ python single_threaded.py
$ python single_threaded.py
Time taken in seconds - 6.20024037361145
Time taken in seconds - 6.20024037361145

Now I modified the code a bit to do to the same countdown using two threads in parallel:

现在，我使用两个并行线程对代码进行了一些修改，以实现相同的倒计时：

And when I ran it again:

当我再次运行它时：

 $ python multi_threaded.py
$ python multi_threaded.py
Time taken in seconds - 6.924342632293701
Time taken in seconds - 6.924342632293701

As you can see, both versions take almost same amount of time to finish. In the multi-threaded version the GIL prevented the CPU-bound threads from executing in parellel.

如您所见，两个版本花费的时间几乎相同。 在多线程版本中，GIL阻止CPU绑定的线程并行执行。

The GIL does not have much impact on the performance of I/O-bound multi-threaded programs as the lock is shared between threads while they are waiting for I/O.

GIL对受I / O绑定的多线程程序的性能影响不大，因为线程在等待I / O时共享锁。

But a program whose threads are entirely CPU-bound, e.g., a program that processes an image in parts using threads, would not only become single threaded due to the lock but will also see an increase in execution time, as seen in the above example, in comparison to a scenario where it was written to be entirely single-threaded.

但是，如上例所示，线程完全受CPU约束的程序（例如使用线程处理映像的程序）不仅会由于锁定而变为单线程，而且执行时间也会增加。与将其编写为完全单线程的方案相比。

This increase is the result of acquire and release overheads added by the lock.

这种增加是锁增加了获取和释放开销的结果。

为什么还没有删除GIL？ (Why hasn’t the GIL been removed yet?)

The developers of Python receive a lot of complaints regarding this but a language as popular as Python cannot bring a change as significant as the removal of GIL without causing backward incompatibility issues.

Python的开发人员对此有很多抱怨，但是像Python这样流行的语言在不引起向后不兼容的问题的情况下，不能带来与删除GIL一样重大的变化。

The GIL can obviously be removed and this has been done multiple times in the past by the developers and researchers but all those attempts broke the existing C extensions which depend heavily on the solution that the GIL provides.

GIL显然可以删除，并且开发人员和研究人员过去已经做过多次，但是所有这些尝试都破坏了现有的C扩展，这些扩展在很大程度上取决于GIL提供的解决方案。

Of course, there are other solutions to the problem that the GIL solves but some of them decrease the performance of single-threaded and multi-threaded I/O-bound programs and some of them are just too difficult. After all, you wouldn’t want your existing Python programs to run slower after a new version comes out, right?

当然，对于GIL解决的问题，还有其他解决方案，但是其中一些解决方案降低了单线程和多线程I / O绑定程序的性能，其中有些太难了。 毕竟，您不希望新版本发布后现有的Python程序运行速度变慢，对吧？

The creator and BDFL of Python, Guido van Rossum, gave an answer to the community in September 2007 in his article “It isn’t Easy to remove the GIL”:

Python的创建者和BDFL的Guido van Rossum在2007年9月的文章“删除GIL并不容易”中向社区做出了回答：

“I’d welcome a set of patches into Py3k only if the performance for a single-threaded program (and for a multi-threaded but I/O-bound program) does not decrease”

“只有在单线程程序（以及多线程但受I / O绑定的程序）的性能不降低的情况下，我才欢迎在Py3k中安装一组补丁程序”

And this condition hasn’t been fulfilled by any of the attempts made since.

此后的任何尝试都未满足此条件。

为什么在Python 3中未将其删除？ (Why wasn’t it removed in Python 3?)

Python 3 did have a chance to start a lot of features from scratch and in the process, broke some of the existing C extensions which then required changes to be updated and ported to work with Python 3. This was the reason why the early versions of Python 3 saw slower adoption by the community.

Python 3确实有机会从头开始启动许多功能，并且在此过程中破坏了一些现有的C扩展，这些扩展随后需要进行更新并移植到Python 3才能使用。这就是早期版本的原因。 Python 3的社区采用速度较慢。

But why wasn’t GIL removed alongside?

但是为什么不将GIL一起删除呢？

Removing the GIL would have made Python 3 slower in comparison to Python 2 in single-threaded performance and you can imagine what that would have resulted in. You can’t argue with the single-threaded performance benefits of the GIL. So the result is that Python 3 still has the GIL.

与单线程性能相比，删除GIL会使Python 3的速度比Python 2慢，并且您可以想象会导致什么。您无法反对GIL的单线程性能的好处。 因此，结果是Python 3仍然具有GIL。

But Python 3 did bring a major improvement to the existing GIL—

但是Python 3确实对现有GIL进行了重大改进-

We discussed the impact of GIL on “only CPU-bound” and “only I/O-bound” multi-threaded programs but what about the programs where some threads are I/O-bound and some are CPU-bound?

我们讨论了GIL对“仅与CPU绑定”和“仅与I / O绑定”的多线程程序的影响，但是其中某些线程受I / O绑定而某些线程与CPU绑定的程序又如何呢？

In such programs, Python’s GIL was known to starve the I/O-bound threads by not giving them a chance to acquire the GIL from CPU-bound threads.

在这样的程序中，众所周知，Python的GIL会给I / O绑定线程带来饥饿，因为它们没有机会从CPU绑定线程获取GIL。

This was because of a mechanism built into Python that forced threads to release the GIL after a fixed interval of continuous use and if nobody else acquired the GIL, the same thread could continue its use.

这是因为Python内置了一种机制，该机制强制线程在固定的连续使用时间间隔后释放GIL，如果没有其他人获得GIL，则同一线程可以继续使用它。

The problem in this mechanism was that most of the time the CPU-bound thread would reacquire the GIL itself before other threads could acquire it. This was researched by David Beazley and visualizations can be found here.

这种机制的问题在于，在大多数情况下，CPU绑定线程会在其他线程无法获取GIL之前重新获取GIL本身。 这是David Beazley进行的研究，可以在此处找到可视化效果。

This problem was fixed in Python 3.2 in 2009 by Antoine Pitrou who added a mechanism of looking at the number of GIL acquisition requests by other threads that got dropped and not allowing the current thread to reacquire GIL before other threads got a chance to run.

这个问题在2009年的Python 3.2中由Antoine Pitrou修复，他添加了一种机制来查看被其他线程丢弃的GIL获取请求的数量，并且不允许当前线程在其他线程有机会运行之前重新获取GIL。

如何处理Python的GIL (How to deal with Python’s GIL)

If the GIL is causing you problems, here a few approaches you can try:

如果GIL给您造成问题，请尝试以下几种方法：

Multi-processing vs multi-threading: The most popular way is to use a multi-processing approach where you use multiple processes instead of threads. Each Python process gets its own Python interpreter and memory space so the GIL won’t be a problem. Python has a multiprocessing module which lets us create processes easily like this:

多处理与多线程：最流行的方法是使用多处理方法，其中您使用多个进程而不是线程。 每个Python进程都有自己的Python解释器和内存空间，因此GIL不会成为问题。 Python有一个multiprocessing模块，使我们可以轻松地创建如下过程：

 from from multiprocessing multiprocessing import import Pool
Pool
import import timetimeCOUNT COUNT = = 50000000
50000000
def def countdowncountdown (( nn ):):while while nn >> 00 ::n n -= -= 11if if __name__ __name__ == == '__main__''__main__' ::pool pool = = PoolPool (( processesprocesses == 22 ))start start = = timetime .. timetime ()()r1 r1 = = poolpool .. apply_asyncapply_async (( countdowncountdown , , [[ COUNTCOUNT //// 22 ])])r2 r2 = = poolpool .. apply_asyncapply_async (( countdowncountdown , , [[ COUNTCOUNT //// 22 ])])poolpool .. closeclose ()()poolpool .. joinjoin ()()end end = = timetime .. timetime ()()printprint (( 'Time taken in seconds -''Time taken in seconds -' , , end end - - startstart )
)

Running this on my system gave this output:

在我的系统上运行此输出：

A decent performance increase compared to the multi-threaded version, right?

与多线程版本相比，性能提高了，对吗？

The time didn’t drop to half of what we saw above because process management has its own overheads. Multiple processes are heavier than multiple threads, so, keep in mind that this could become a scaling bottleneck.

时间并没有减少到我们上面看到的一半，因为流程管理有其自己的开销。 多个进程比多个线程重，因此请记住，这可能会成为扩展瓶颈。

Alternative Python interpreters: Python has multiple interpreter implementations. CPython, Jython, IronPython and PyPy, written in C, Java, C# and Python respectively, are the most popular ones. GIL exists only in the original Python implementation that is CPython. If your program, with its libraries, is available for one of the other implementations then you can try them out as well.

可选的Python解释器： Python具有多种解释器实现。 最受欢迎的分别是用C，Java，C＃和Python编写的CPython，Jython，IronPython和PyPy。 GIL仅存在于原始Python实现中，即CPython。 如果您的程序及其库可用于其他实现之一，则也可以尝试一下。

Just wait it out: While many Python users take advantage of the single-threaded performance benefits of GIL. The multi-threading programmers don’t have to fret as some of the brightest minds in the Python community are working to remove the GIL from CPython. One such attempt is known as the Gilectomy.

请稍等：尽管许多Python用户利用GIL的单线程性能优势。 多线程程序员不必烦恼，因为Python社区中一些最聪明的人正在努力从CPython中删除GIL。 一种这样的尝试被称为“ Gilectomy” 。

The Python GIL is often regarded as a mysterious and difficult topic. But keep in mind that as a Pythonista you’re usually only affected by it if you are writing C extensions or if you’re using CPU-bound multi-threading in your programs.

Python GIL通常被认为是一个神秘而困难的话题。 但是请记住，作为Pythonista，通常只有在编写C扩展或在程序中使用CPU绑定多线程时才受到它的影响。

In that case, this article should give you everything you need to understand what the GIL is and how to deal with it in your own projects. And if you want to understand the low-level inner workings of GIL, I’d recommend you watch the Understanding the Python GIL talk by David Beazley.

在这种情况下，本文应为您提供了解GIL以及如何在您自己的项目中处理GIL所需的一切。 而且，如果您想了解GIL的底层内部工作原理，建议您观看David Beazley的“ 了解Python GIL”演讲。

翻译自: https://www.pybloggers.com/2018/03/what-is-the-python-global-interpreter-lock-gil/

python gil全局锁