Python上下文语法with小述

本文环境python3.5.2

上下文语法with

该语法糖主要便于在Python的编程过程中，能够有效管理防止编程过程中，对有关资源编程时忘记释放的问题，比如通过with来open一个文件，就不需要显式的在处理完成文件之后调用f.close方法，易于简洁编写相关代码。在Python中通过contextlib提供了两种的上下文的实现方式，分别是通过继承ContextDecorator的类的实现，通过contextmanager装饰器函数的实现。

with的上下文语法的实现方式

无论是哪种实现方式，都是通过调用了类的__enter__和__exit__函数来实现的上下文管理，这在Python的官方文档中也做了相关定义介绍，无论是通过ContextDecorator还是通过contextmanager都是基于该两个内建方法进行上下文文法的实现的。首先分析如下代码：

from contextlib import ContextDecorator, contextmanagerclass WithClass(ContextDecorator):def __enter__(self):print('__enter__')return selfdef __exit__(self, exc_type, exc_val, exc_tb):print('__exit__')def test_print(self):print("test_print")with WithClass() as w:w.test_print()

with对应的字节码模块的内容分析

我们首先分析一下该代码的生成对应的字节码如下：

  2           0 LOAD_CONST               0 (0)3 LOAD_CONST               1 (('ContextDecorator', 'contextmanager'))6 IMPORT_NAME              0 (contextlib)9 IMPORT_FROM              1 (ContextDecorator)12 STORE_NAME               1 (ContextDecorator)15 IMPORT_FROM              2 (contextmanager)18 STORE_NAME               2 (contextmanager)21 POP_TOP4          22 LOAD_BUILD_CLASS23 LOAD_CONST               2 (<code object WithClass at 0x10805ff60, file "<dis>", line 4>)26 LOAD_CONST               3 ('WithClass')29 MAKE_FUNCTION            032 LOAD_CONST               3 ('WithClass')35 LOAD_NAME                1 (ContextDecorator)38 CALL_FUNCTION            3 (3 positional, 0 keyword pair)41 STORE_NAME               3 (WithClass)16          44 LOAD_NAME                3 (WithClass)47 CALL_FUNCTION            0 (0 positional, 0 keyword pair)50 SETUP_WITH              17 (to 70)53 STORE_NAME               4 (w)17          56 LOAD_NAME                4 (w)59 LOAD_ATTR                5 (test_print)62 CALL_FUNCTION            0 (0 positional, 0 keyword pair)65 POP_TOP66 POP_BLOCK67 LOAD_CONST               4 (None)>>   70 WITH_CLEANUP_START71 WITH_CLEANUP_FINISH72 END_FINALLY73 LOAD_CONST               4 (None)76 RETURN_VALUE

通过生成的字节码可知，16行和17行就是对with语言的解析，首先先加载一个名称WithClass，然后通过CALL_FUNCTION来调用WithClass类的__call__方法，接着便是调用了SETUP_WITH来生成一个对象并把它赋值给变量w，在17行就获取w然后调用了w的属性test_print方法，在17行执行完成后，就调用了WITH_CLEANUP_START和WITH_CLEANUP_FINISH方法其实就是调用了__exit__方法对应的函数，最后调用了END_FINALLY，至此一个with的语法就执行完成。

其中SETUP_WITH对应的字节码如下；

    TARGET(SETUP_WITH) {_Py_IDENTIFIER(__exit__);                                                # 获取__exit__的名称_Py_IDENTIFIER(__enter__);                                               # 获取__enter__的名称PyObject *mgr = TOP();                                                     # 获取对应实例化的类PyObject *exit = special_lookup(mgr, &PyId___exit__), *enter;           # 先通过实例类去查找__exit__方法PyObject *res;if (exit == NULL)                                                      # 如果exit为空则报错goto error;SET_TOP(exit);                                                          # 把获取的exit方法放置在栈顶enter = special_lookup(mgr, &PyId___enter__);                             # 通过类查找__enter__方法Py_DECREF(mgr); if (enter == NULL)                                                      # 如果为空则报错goto error;res = PyObject_CallFunctionObjArgs(enter, NULL);                       # 调用查找到的enter函数并传入null参数执行Py_DECREF(enter);if (res == NULL)                                                       # 如果执行的enter函数的返回值为空则报错goto error;/* Setup the finally block before pushing the resultof __enter__ on the stack. */PyFrame_BlockSetup(f, SETUP_FINALLY, INSTR_OFFSET() + oparg,STACK_LEVEL());PUSH(res);                                                               # 压入返回值DISPATCH();}

由此可知，首先就执行了对应类实例的__enter__函数，并获取了返回值，然后执行完成with包裹的内容之后就调用WITH_CLEANUP_START来清理包裹的所有的数据并调用exit函数；

    TARGET(WITH_CLEANUP_START) {/* At the top of the stack are 1-6 values indicatinghow/why we entered the finally clause:- TOP = None- (TOP, SECOND) = (WHY_{RETURN,CONTINUE}), retval- TOP = WHY_*; no retval below it- (TOP, SECOND, THIRD) = exc_info()(FOURTH, FITH, SIXTH) = previous exception for EXCEPT_HANDLERBelow them is EXIT, the context.__exit__ bound method.In the last case, we must callEXIT(TOP, SECOND, THIRD)otherwise we must callEXIT(None, None, None)In the first three cases, we remove EXIT from thestack, leaving the rest in the same order.  In thefourth case, we shift the bottom 3 values of thestack down, and replace the empty spot with NULL.In addition, if the stack represents an exception,*and* the function call returns a 'true' value, wepush WHY_SILENCED onto the stack.  END_FINALLY willthen not re-raise the exception.  (But non-localgotos should still be resumed.)*/PyObject *exit_func;PyObject *exc = TOP(), *val = Py_None, *tb = Py_None, *res;            # 检查执行完成信息if (exc == Py_None) {                                                   # 如果没有执行报错(void)POP();                                                  exit_func = TOP();                                                     # 获取exit函数      SET_TOP(exc);                                                       # 把执行信息置顶}else if (PyLong_Check(exc)) {                                             # 检查执行结果STACKADJ(-1);switch (PyLong_AsLong(exc)) {case WHY_RETURN:case WHY_CONTINUE:/* Retval in TOP. */exit_func = SECOND();SET_SECOND(TOP());SET_TOP(exc);break;default:exit_func = TOP();SET_TOP(exc);break;}exc = Py_None;}else {PyObject *tp2, *exc2, *tb2;PyTryBlock *block;val = SECOND();tb = THIRD();tp2 = FOURTH();exc2 = PEEK(5);tb2 = PEEK(6);exit_func = PEEK(7);SET_VALUE(7, tb2);SET_VALUE(6, exc2);SET_VALUE(5, tp2);/* UNWIND_EXCEPT_HANDLER will pop this off. */SET_FOURTH(NULL);/* We just shifted the stack down, so we haveto tell the except handler block that thevalues are lower than it expects. */block = &f->f_blockstack[f->f_iblock - 1];assert(block->b_type == EXCEPT_HANDLER);block->b_level--;}/* XXX Not the fastest way to call it... */res = PyObject_CallFunctionObjArgs(exit_func, exc, val, tb, NULL);                # 调用exit函数并将执行的结果传入执行Py_DECREF(exit_func);if (res == NULL)                                                                # 如果res为空则报错goto error;Py_INCREF(exc); /* Duplicating the exception on the stack */PUSH(exc);PUSH(res);PREDICT(WITH_CLEANUP_FINISH);DISPATCH();}

此处的两个字节码执行就是主要的with上下文所做的主要内容，主要还是依赖于对象的__exit__与__enter__两个方法。

ContextDecorator的实现的简单浅析

由上一节的内容可知，with上下文主要依赖于__enter__和__exit__两个方法的实现，所以查看一下ContextDecorator的代码；

class ContextDecorator(object):"A base class or mixin that enables context managers to work as decorators."def _recreate_cm(self):"""Return a recreated instance of self.Allows an otherwise one-shot context manager like_GeneratorContextManager to support use asa decorator via implicit recreation.This is a private interface just for _GeneratorContextManager.See issue #11647 for details."""return self                                         # 返回自己def __call__(self, func):                               # 如果使用了装饰器来包装被调用函数@wraps(func)def inner(*args, **kwds):               with self._recreate_cm():                       # 调用self的__enter__和__exit__方法return func(*args, **kwds)                  # 然后调用被包装函数执行return inner

所以在上述示例代码中的打印函数也可改写成如下；

@WithClass()
def test_print_func():print("test_print")test_print_func()

上述内容基本上就是ContextDecorator的全部内容，相对比较简单易于理解。

contextmanager的函数形式的上下文实现

def contextmanager(func):"""@contextmanager decorator.Typical usage:@contextmanagerdef some_generator(<arguments>):<setup>try:yield <value>finally:<cleanup>This makes this:with some_generator(<arguments>) as <variable>:<body>equivalent to this:<setup>try:<variable> = <value><body>finally:<cleanup>"""@wraps(func)def helper(*args, **kwds):return _GeneratorContextManager(func, args, kwds)       # 装饰器生成_GeneratorContextManager类实例return helper

由函数的注释可知，该函数实现的上下文依赖于协程来实现的，该函数的定义相对简单，直接生成_GeneratorContextManager类实例，

class _GeneratorContextManager(ContextDecorator):"""Helper for @contextmanager decorator."""def __init__(self, func, args, kwds):self.gen = func(*args, **kwds)                                          # 生成协程函数self.func, self.args, self.kwds = func, args, kwds                      # 保存函数与传入参数等值# Issue 19330: ensure context manager instances have good docstringsdoc = getattr(func, "__doc__", None)                                    # 获取函数文档说明if doc is None:doc = type(self).__doc__self.__doc__ = doc                                                      # 设置文档说明# Unfortunately, this still doesn't provide good help output when# inspecting the created context manager instances, since pydoc# currently bypasses the instance docstring and shows the docstring# for the class instead.# See http://bugs.python.org/issue19404 for more details.def _recreate_cm(self):# _GCM instances are one-shot context managers, so the# CM must be recreated each time a decorated function is# calledreturn self.__class__(self.func, self.args, self.kwds)                  # 实例化自己def __enter__(self):try:return next(self.gen)                                               # 获取fun中yield返回的值except StopIteration:raise RuntimeError("generator didn't yield") from Nonedef __exit__(self, type, value, traceback):if type is None:try:next(self.gen)                                                  # 如果执行没有错误则继续nextexcept StopIteration:return                                                          # 如果是StopIteration则立即返回else:raise RuntimeError("generator didn't stop")                     # 否则返回生成器错误else:if value is None:# Need to force instantiation so we can reliably# tell if we get the same exception backvalue = type()try:self.gen.throw(type, value, traceback)raise RuntimeError("generator didn't stop after throw()")except StopIteration as exc:# Suppress StopIteration *unless* it's the same exception that# was passed to throw().  This prevents a StopIteration# raised inside the "with" statement from being suppressed.return exc is not valueexcept RuntimeError as exc:# Likewise, avoid suppressing if a StopIteration exception# was passed to throw() and later wrapped into a RuntimeError# (see PEP 479).if exc.__cause__ is value:return Falseraiseexcept:# only re-raise if it's *not* the exception that was# passed to throw(), because __exit__() must not raise# an exception unless __exit__() itself failed.  But throw()# has to raise the exception to signal propagation, so this# fixes the impedance mismatch between the throw() protocol# and the __exit__() protocol.#if sys.exc_info()[1] is not value:raise

由上述代码可知，该_GeneratorContextManager所包装的函数必须是生成器函数，其实现基于ContextDecorator类的实现思路实现。改造的示例代码如下；

@contextmanager
def test_func():w = WithClass()try:yield wfinally:print("test_func finally")with test_func() as f:f.test_print()

总结

本文的有关Python上下文语法分析的内容短小，相应的使用的规则并不复杂，主要还是通过实现__enter__和__exit__两个方法来实现上下文，达到便捷的编程，contextlib中还有其他模块可供使用，大家有兴趣可自行查阅。鉴于本人才疏学浅，如有疏漏请批评指正。