python和c混合编程 gil,如何在python中使用C扩展来解决GIL

I want to run a cpu intensive program in Python across multiple cores and am trying to figure out how to write C extensions to do this. Are there any code samples or tutorials on this?

解决方案

You can already break a Python program into multiple processes. The OS will already allocate your processes across all the cores.

Do this.

python part1.py | python part2.py | python part3.py | ... etc.

The OS will assure that part uses as many resources as possible. You can trivially pass information along this pipeline by using cPickle on sys.stdin and sys.stdout.

Without too much work, this can often lead to dramatic speedups.

Yes -- to the haterz -- it's possible to construct an algorithm so tortured that it may not be sped up much. However, this often yields huge benefits for minimal work.

And.

The restructuring for this purpose will exactly match the restructuring required to maximize thread concurrency. So. Start with shared-nothing process parallelism until you can prove that sharing more data would help, then move to the more complex shared-everything thread parallelism.

python和c混合编程 gil,如何在python中使用C扩展来解决GIL相关推荐

python列表可以混合类型_如何在Python中将混合数据类型的列表转换为数据帧
我有一个混合数据类型列表,如下所示:list = [['3D prototypes', 'Can print large objects', 'Autodesk Maya/Mudbox', '3D S ...
c python boost.python_如何利用Boost.Python实现Python C/C++混合编程详解
前言学习中如果碰到问题,参考官网例子:D:boost_1_61_0libspythontest 参考:Boost.Python 中英文文档. 利用Boost.Python实现Python C/C++ ...
python与matlab混合编程_python 与 matlab 混编
Matlab的官方文档中介绍了 Matlab 与其余编程语言之间的引擎接口,其中包括对于 Python 开放的引擎 API,可参考官方教程,其中包括引擎安装,基本使用,以及Python与Matlab之 ...
Boost.Python实现Python C/C++混合编程
导出函数 #include<string> #include<boost/python.hpp>using namespace std; using namespace boo ...
cuda编程python接口_混合编程[python+cpp+cuda]
很多时候,我们是基于python进行模型的设计和运行,可是基于python本身的速度问题,使得原生态python代码无法满足生产需求,不过我们可以借助其他编程语言来缓解python开发的性能瓶颈.这里 ...
C++和Python混合编程：C++调用Python函数
文章目录一.C++直接运行python代码的控制台Demo 二.环境配置三.C++调用Python函数 C++传入Python的参数格式转换 C++调用Python[有参有返回值]函数 C++运行 ...
python怎么去掉换行符_如何在Python中删除尾部换行符？
如何在Python中删除尾部换行符? 什么是Perl的chomp函数的Python等价物,如果它是换行符,它会删除字符串的最后一个字符? 26个解决方案 1473 votes 尝试方法lstrip() ...
Python要了解哪些编程基础如何学Python比较好
Python要了解哪些编程基础?如何学Python比较好?不管怎么说,Python都是大家进军IT行业值得选择的语言.毕竟它不但使用,而且还好用.更值得一提的是,它非常容易入门.而且在人工智能.传统编 ...
python 参数个数同名函数_如何在python中编写不同参数的同名方法
我在Java背景下学习Python(3.x). 我有一个python程序,我在其中创建一个personObject并将其添加到列表中.p = Person("John") list ...

python和c混合编程 gil,如何在python中使用C扩展来解决GIL

python和c混合编程 gil,如何在python中使用C扩展来解决GIL相关推荐

最新文章

热门文章