sqlalchemy 系列教程五-Session和scopedsession

sqlalchemy 中 session 是什么, scoped_session 又是什么 ?

sqlalchemy 中的 session , scoped_session 的区别

摘要 : 本文主要讲解 Session 是什么, scopedsession

sqlalchemy 中的 session , scoped_session 的区别是什么?
如何进行session 管理

Session 其实就是一个会话, 可以和数据库打交道的一个会话.

官方是这样说明的.

The orm.mapper() function and declarative extensions are the primary configurational interface for the ORM. Once mappings are configured, the primary usage interface for persistence operations is the Session.

orm.mapper（）函数和声明性扩展是ORM的主要配置接口。配置映射后，持久性操作的主要用法接口是Session.

即用session 来操作数据库. 通过 session 来实现增删改查的操作.

为了方便说明首先先定义一个base

#base.py  from datetime import datetimefrom secure import XINYONGFEI_BI_URL
from sqlalchemy import Column, Integer, Stringfrom sqlalchemy import create_engine, DateTime
from sqlalchemy.ext.declarative import declarative_baseengine = create_engine(XINYONGFEI_BI_URL, pool_size=10, pool_recycle=7200,pool_pre_ping=True, encoding='utf-8')_Base = declarative_base(bind=engine, name='base')class Base(_Base):__abstract__ = True__table_args__ = {'mysql_engine': 'InnoDB','mysql_charset': 'utf8',"useexisting": True}create_time = Column(DateTime, nullable=False, default=datetime.now)

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
@Time    : 2019/5/5 18:21
@File    : person.py
@Author  : frank.chang@shoufuyou.com
"""from base import Base
from sqlalchemy import Column, Integer, Stringclass Person(Base):__tablename__ = 'Person'id = Column(Integer, primary_key=True, autoincrement=True)name = Column(String(length=64), comment='姓名')mobile = Column(String(length=13), comment='手机号')id_card_number = Column(String(length=64), comment='身份证')def __str__(self):return '%s(name=%r,mobile=%r,id_card_number=%r)' % (self.__class__.__name__,self.name,self.mobile,self.id_card_number)__repr__ = __str__if __name__ == '__main__':person = Person(name='frank-' + 'job4', mobile='4444444444', id_card_number='123456789')person = Person(name='frank-' + 'job4', mobile='4444444444', id_card_number='123456789')person1 = Person(name='frank-' + 'job1', mobile='111111111', id_card_number='11111111111')print(person)pass

首先创建一个session ,操作一下 Session 对象

>>> from sqlalchemy.orm import sessionmaker, scoped_session>>> session_factory = sessionmaker(bind=engine)
...
>>>
>>>
>>> person = Person(name='frank-' + 'job3', mobile='111111', id_card_number='123456789')
...
>>> session= session_factory()
>>> s1= session_factory()
>>> s1
<sqlalchemy.orm.session.Session object at 0x107ec8c18>
>>> s2 = session_factory()
>>> s1,s2
(<sqlalchemy.orm.session.Session object at 0x107ec8c18>, <sqlalchemy.orm.session.Session object at 0x107ee3ac8>)
>>> s1 is s2
False
>>> id(s1),id(s2)
(4427910168, 4428020424)>>> s1.add(person)
>>> s2.add(person)
Traceback (most recent call last):File "<input>", line 1, in <module>File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1835, in addself._save_or_update_state(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1848, in _save_or_update_stateself._save_or_update_impl(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2163, in _save_or_update_implself._save_impl(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2115, in _save_implto_attach = self._before_attach(state, obj)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2238, in _before_attachstate.session_id, self.hash_key))
sqlalchemy.exc.InvalidRequestError: Object '<Person at 0x107ec8128>' is already attached to session '11' (this is '12')

当我使用 s2 添加 person 就会报错了. 说person 这个对象已经和另一个session 关联一起来了, 再次关联另一个session 就会报错.

如果s1 ,s2 关联不同的person就不会有问题.


>>> person4 = Person(name='frank-' + 'job4', mobile='4444444444', id_card_number='123456789')
>>> person1 = Person(name='frank-' + 'job1', mobile='111111', id_card_number='123456789')
>>>
>>> s1.rollback()
>>> s1.add(person1)
>>> s2.add(person4)
>>> s1.commit()  # 提交数据
>>> s2.commit()  # 提交数据, 写入数据库

当执行了 commit 操作的后, 数据才会写入数据库. 此时数据库里面已经有了数据.

即不同的session 是关联不同的表, 一个session 如果有关联一个从base 继承的对象, 其他的session 是不能够关联这个对象的. 就相当于把 session 和对象绑定在一起了.
此时数据库里面就有两条数据了.

#对同一个对象提交多次,commit 多次,也只会有一条数据写入数据库.

提交多次不同的对象可以写入多条元素.

为了方便演示

def get_session_no_scope():engine = create_engine(XINYONGFEI_BI_URL, pool_size=5, pool_recycle=7200,pool_pre_ping=True, encoding='utf-8')session_factory = sessionmaker(bind=engine)session = session_factory()return session>>> p= Person(name='frank', mobile='11111111',id_card_number= '123456789')
>>>
>>> s1 = get_session_no_scope()
>>> s1.add(p)
>>> s1.add(p)
>>> s1.add(p)
>>> s1.commit()

commit 之后数据库里面会有一条记录,并不会有三条记录.
而且此时如果 ,不进行 close,或者回滚, 其他的session 是不能添加 p 这个对象的.


s2 = get_session_no_scope()
s2.add(p)
Traceback (most recent call last):File "<input>", line 1, in <module>File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1835, in addself._save_or_update_state(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1848, in _save_or_update_stateself._save_or_update_impl(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2165, in _save_or_update_implself._update_impl(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2148, in _update_implto_attach = self._before_attach(state, obj)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2238, in _before_attachstate.session_id, self.hash_key))
sqlalchemy.exc.InvalidRequestError: Object '<Person at 0x10e677f28>' is already attached to session '5' (this is '6')
s1.close()
s2.add(p)
s2.commit()

s1.close() 之后, s2 就可以顺利 add , commit了,但是此时并没有新生成一条数据,到数据库里面. 难道同一个对象只能 commit 一次. 但是此时数据库的数据并没有新增.

测试发现同一对象, 只能提交到数据库一次.

engine = create_engine(XINYONGFEI_BI_URL, pool_size=5, pool_recycle=7200,pool_pre_ping=True, encoding='utf-8')session_factory = sessionmaker(bind=engine)
Session = scoped_session(session_factory)def get_session_no_scope():return session_factory()

>>> s1 = get_session_no_scope()
>>> person = Person(name='frank-' + 'job1', mobile='111111', id_card_number='123456789')
...
>>> person
Person(name='frank-job1',mobile='111111',id_card_number='123456789')
>>> s1.add(person)
>>> s1.commit()
>>> s1.add(person)
>>> s1.commit()
>>> s1.close()

此时数据库里面会增加一条数据 , 不会增加两条数据. 并且一旦session.add(person), 其他的session 是不能操作这个对象的. 除非 session.close() 才能有其他的session 操作这个对象.

到此大概清楚了session 是主要给用户提供操作数据库的接口, 用来进行增删改查操作.
给用户提供比较高级的API ,来操作数据库, 并且并且同一个session 只能操作一个对象,

总结:
假设有两个session分别为 s1,s2 , s1.add(person),之后 s2 是不能操作 person 的.
直到 s1.close() ,s2 才能操作person

del session, session2
s1 =session_factory()
s2 =session_factory()
s1,s2
(<sqlalchemy.orm.session.Session object at 0x1115586d8>, <sqlalchemy.orm.session.Session object at 0x112030d30>)p= Person(name='frank-' + 'job4', mobile='4444444444', id_card_number='123456789')p.name='frank111'
p
Person(name='frank111',mobile='4444444444',id_card_number='123456789')
s1.add(p)
s1.rollback()
s2.add(p)
s2.rollback()
s1.add(p)
s1.commit()
s2.add(p)
Traceback (most recent call last):File "<input>", line 1, in <module>File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1947, in addself._save_or_update_state(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1960, in _save_or_update_stateself._save_or_update_impl(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2303, in _save_or_update_implself._update_impl(state)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2286, in _update_implto_attach = self._before_attach(state, obj)File "/Users/frank/.local/share/virtualenvs/mysqlalchemy-demo-0htClb7e/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2374, in _before_attach% (state_str(state), state.session_id, self.hash_key)
sqlalchemy.exc.InvalidRequestError: Object '<Person at 0x1120ff208>' is already attached to session '6' (this is '7')
s1.close()
s2.add(p)
s2.commit()

2 scoped_session 是什么?

用scope_session 生成的对象

engine = create_engine(XINYONGFEI_BI_URL, pool_size=5, pool_recycle=7200,pool_pre_ping=True, encoding='utf-8')session_factory = sessionmaker(bind=engine)
Session = scoped_session(session_factory)def get_session():return Session()

>>>
>>> s3= get_session()
>>> p2 = Person(name='frank-2', mobile='222222', id_card_number='123456789')
>>> s4= get_session()>>> s3 is s4
True
>>> s3.add(p2)
>>> s4.add(p2)
>>> s4.commit()

发现就可以用两个session 同时操作一个对象. 打印 id 发现其实生成的是一个session 同一个id .

这个感觉有点像单例模式, 只有一个对象, 一个session 对象.

实际上不是这样的. scoped_session 的作用是为了多线程下面共享 session .

从scoped_session对象生成的session 对象，然后我们没有这样的问题，因为scoped_session对象维护同一会话对象的注册表。

官方文档对scopesession 介绍如下 https://docs.sqlalchemy.org/en/13/orm/contextual.html#unitofwork-contextual

refer: https://docs.sqlalchemy.org/en/13/orm/contextual.html#unitofwork-contextualThread-Local Scope
Users who are familiar with multithreaded programming will note that representing anything as a global variable is usually a bad idea, as it implies that the global object will be accessed by many threads concurrently. The Session object is entirely designed to be used in a non-concurrent fashion, which in terms of multithreading means “only in one thread at a time”. So our above example of scoped_session usage, where the same Session object is maintained across multiple calls, suggests that some process needs to be in place such that multiple calls across many threads don’t actually get a handle to the same session. We call this notion thread local storage, which means, a special object is used that will maintain a distinct object per each application thread. Python provides this via the threading.local() construct. The scoped_session object by default uses this object as storage, so that a single Session is maintained for all who call upon the scoped_session registry, but only within the scope of a single thread. Callers who call upon the registry in a different thread get a Session instance that is local to that other thread.Using this technique, the scoped_session provides a quick and relatively simple (if one is familiar with thread-local storage) way of providing a single, global object in an application that is safe to be called upon from multiple threads.The scoped_session.remove() method, as always, removes the current Session associated with the thread, if any. However, one advantage of thethreading.local() object is that if the application thread itself ends, the “storage” for that thread is also garbage collected. So it is in fact “safe” to use thread local scope with an application that spawns and tears down threads, without the need to call scoped_session.remove(). However, the scope of transactions themselves, i.e. ending them via Session.commit() or Session.rollback(), will usually still be something that must be explicitly arranged for at the appropriate time, unless the application actually ties the lifespan of a thread to the lifespan of a transaction.

熟悉多线程编程的用户都知道任何东西作为一个全局变量不是一个好主意. 这意味着这个全局的对象可以被多线程并发的访问. 而 Session 这个对象设计的时候仅仅使用在非并发场景, 这意味着同一时刻只用一个线程. 上面所举例子, scoped_session ,多次调用中维护相同的 Session 对象. 这表明, 在某些进程中需要一个适当的位置 ,就像多线程调用不会获得相同的 session 句柄. 我们把这个概念叫做线程本地存储. 这意味着使用一个特殊的对象 , 为每个应用线程保持着一个不同的对象.

python 提供了 threading.local 构造器. scoped_session 对象默认是使用这个对象作为存储,这样就可以为所有调用scoped_session注册表的对象维护一个Session,但只能在单个线程的范围内。在不同线程中根据注册表的调用这个可以得到一个session 对象, 这个session 对象对于其他的线程是 local 的. (理解: 就是说不同的线程拿到的session 是不一样的.)

通过这种方式, scoped_session提供了一个快速并且相对简单的提供单个session 方式. 对于多线程的调用来说, 全局对象是安全的.

scoped_session.remove 方法 , 总是, 移除当前的session 和与之关联的线程的联系. 然而, 事务的范围, 通过session.commit() or sesson.rollback() 来结束. 并且事务的范围在合适的时间可以清晰的管理 , 除非应用确实希望线程的生命周期和事务的生命周期紧紧地绑定在一起.

用多线程的方式来看下 scope_session 到底如何使用的.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
@Time    : 2019/5/5 18:33
@File    : test_session.py
@Author  : frank.chang@shoufuyou.com# person = Person(name='frank-' + 'job1', mobile='111111', id_card_number='123456789')
session = get_scoped_session()"""import threading
import timefrom sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, scoped_session
from secure import XINYONGFEI_BI_URLfrom model.person import Personengine = create_engine(XINYONGFEI_BI_URL, pool_size=5, pool_recycle=7200,pool_pre_ping=True, encoding='utf-8')session_factory = sessionmaker(bind=engine)
Session = scoped_session(session_factory)def get_scoped_session():"""scoped_session:return:"""return Session()def get_session_no_scope():""" noscoped_session """return session_factory()session = get_scoped_session()def job(name):global session# 这里打印一下 session id  print(f"id session:{id(session)}")person = Person(name='frank-' + name, mobile='111111', id_card_number='123456789')print(f"{name} person is add..")session.add(person)time.sleep(2)if name == 'job3':# 线程3 提交 , 其他线程不提交.session.commit()if __name__ == '__main__':thread_list = []# 创建5个线程for i in range(5):name = 'job' + str(i)t = threading.Thread(target=job, name=name, args=(name,))thread_list.append(t)for t in thread_list:t.start()for t in thread_list:t.join()

我这里开启5 个线程, 我想让线程3 进行提交 person对象, 其他的线程不提交.

id session:4541434120
job0 person is add..
id session:4541434120
job1 person is add..
id session:4541434120
job2 person is add..
id session:4541434120
job3 person is add..
id session:4541434120
job4 person is add..Process finished with exit code 0

运行结果如下: 可以看出因为使用的是同一个session , id 是一样的. 但是只要有一个线程提交了. 其他的线程也会受到影响. 这个session 相当于被共享了.

如果想要实现线程之间不会相互干扰, 可以每次都生成一个session
每次任务都起来的时候, 重新创建一个session 就可以了.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
@Time    : 2019/5/5 18:33
@File    : test_session.py
@Author  : frank.chang@shoufuyou.com# person = Person(name='frank-' + 'job1', mobile='111111', id_card_number='123456789')
session = get_scoped_session()"""import threading
import timefrom sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, scoped_session
from secure import XINYONGFEI_BI_URLfrom model.person import Personengine = create_engine(XINYONGFEI_BI_URL, pool_size=5, pool_recycle=7200,pool_pre_ping=True, encoding='utf-8')session_factory = sessionmaker(bind=engine)
Session = scoped_session(session_factory)def get_scoped_session():"""scoped_session:return:"""return Session()def get_session_no_scope():""" noscoped_session """return session_factory()def job(name):# 这里每次 创建一个新的session 对象session = get_scoped_session()print(f"id session:{id(session)}")person = Person(name='frank-' + name, mobile='111111', id_card_number='123456789')print(f"{name} person is add..")session.add(person)time.sleep(2)if name == 'job3':# 线程3 提交 , 其他线程不提交.session.commit()session.close()if __name__ == '__main__':thread_list = []# 创建5个线程for i in range(5):name = 'job' + str(i)t = threading.Thread(target=job, name=name, args=(name,))thread_list.append(t)for t in thread_list:t.start()for t in thread_list:t.join()

结果如下:

id session:4584472304
job0 person is add..
id session:4584866200
job1 person is add..
id session:4584866704
job2 person is add..
id session:4584867208
job3 person is add..
id session:4584867936
job4 person is add..

可以看出每次创建的 session_id 都不一样了.

此时数据库里面只有一条数据.

scoped_session 默认情况会根据线程创建不同的session , 同一个线程下面创建的session 是一样的. 不同的线程创建的session 是不一样的.

默认情况下会调用 ThreadLocalRegistry() 这个对象的call 方法,这里做了线程隔离,让不同的线程拿到不同的session.

用session 来重新测试上面的代码.

1 先用全局的session , 这样会导致每条数据都会被写入数据库.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
@Time    : 2019/5/5 18:33
@File    : test_session.py
@Author  : frank.chang@shoufuyou.com# person = Person(name='frank-' + 'job1', mobile='111111', id_card_number='123456789')
session = get_session_no_scope()"""import threading
import timefrom sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, scoped_session
from secure import XINYONGFEI_BI_URLfrom model.person import Personengine = create_engine(XINYONGFEI_BI_URL, pool_size=5, pool_recycle=7200,pool_pre_ping=True, encoding='utf-8')session_factory = sessionmaker(bind=engine)
Session = scoped_session(session_factory)def get_session_no_scope():""" noscoped_session """return session_factory()session = get_session_no_scope()def job(name):global sessionprint(f"id session:{id(session)}")person = Person(name='frank-' + name, mobile='111111', id_card_number='123456789')print(f"{name} person is add..")session.add(person)time.sleep(1)if name == 'job3':# 线程3 提交, 其他线程不提交.session.commit()session.close()if __name__ == '__main__':thread_list = []# 创建5个线程for i in range(5):name = 'job' + str(i)t = threading.Thread(target=job, name=name, args=(name,))thread_list.append(t)for t in thread_list:t.start()for t in thread_list:t.join()

结果如下:

id session:4418463896
job0 person is add..
id session:4418463896
job1 person is add..
id session:4418463896
job2 person is add..
id session:4418463896
job3 person is add..
id session:4418463896
job4 person is add..

图4 所有的数据都会被写入数据库.

看出此时所有的数据都被写入到数据库里面了.

总结

session 和scopedsession 的区别, scoped_session 实现了一个线程的隔离, 保证不同的线程拿到不同的session, 同一个线程拿到的session 是同一个值.
session 和scopedsession 本质上都是用来操作数据库的. 只是session 只适合在单线程下面使用.

参考文档

1 数据库连接池SQLAlchemy中多线程安全的问题 https://blog.csdn.net/daijiguo/article/details/79486294
2 sqlalchemy_session http://rhel.cc/2016/07/14/sqlalchemy-session/
3 session 官方地址 https://docs.sqlalchemy.org/en/13/orm/session.html
4 scoped_session https://docs.sqlalchemy.org/en/13/orm/contextual.html#unitofwork-contextual
5 SQLAlchemy 基础知识 - autoflush 和 autocommit https://zhuanlan.zhihu.com/p/48994990
6 sesssion 官方首页 https://docs.sqlalchemy.org/en/13/orm/session.html
7 sqlalchemy 常见的问题 https://docs.sqlalchemy.org/en/13/orm/session_basics.html#session-faq-whentocreate
8 多线程下面的使用问题 https://blog.csdn.net/daijiguo/article/details/79486294

分享快乐,留住感动. 2019-05-13 19:19:27 --frank