mongoDB VS PostgreSQL dml performance use python (pymongo py-postgresql)
2013-04-27 05:26:14.729 CST,"digoal","",27083,"192.1.1.1:58679",5179c9ad.69cb,4,"sending backup ""pg_basebackup base backup""",2013-04-26 08:26:21 CST,59/0,0,LOG,08006,"could not send data to client: Connection reset by peer",,,,,,,,"internal_flush, pqcomm.c:1253","pg_basebackup"
2013-04-27 05:26:14.729 CST,"digoal","",27083,"192.1.1.1:58679",5179c9ad.69cb,5,"sending backup ""pg_basebackup base backup""",2013-04-26 08:26:21 CST,59/0,0,FATAL,XX000,"base backup could not send data, aborting backup",,,,,,,,"sendFile, basebackup.c:1027","pg_basebackup"
2013-04-27 05:26:14.729 CST,"digoal","",27083,"192.1.1.1:58679",5179c9ad.69cb,6,"sending backup ""pg_basebackup base backup""",2013-04-26 08:26:21 CST,59/0,0,LOG,08006,"could not send data to client: Broken pipe",,,,,,,,"internal_flush, pqcomm.c:1253","pg_basebackup"
2013-04-27 05:26:14.729 CST,"digoal","",27083,"192.1.1.1:58679",5179c9ad.69cb,7,"sending backup ""pg_basebackup base backup""",2013-04-26 08:26:21 CST,59/0,0,FATAL,08006,"connection to client lost",,,,,,,,"ProcessInterrupts, postgres.c:2867","pg_basebackup"
2013-04-27 05:26:14.729 CST,"digoal","",27083,"192.1.1.1:58679",5179c9ad.69cb,8,"sending backup ""pg_basebackup base backup""",2013-04-26 08:26:21 CST,,0,LOG,00000,"disconnection: session time: 20:59:52.949 user=digoal database= host=192.1.1.1 port=58679",,,,,,,,"log_disconnections, postgres.c:4366","pg_basebackup"
/etc/sysctl.conf
net.ipv4.tcp_keepalive_time=90
sysctl -p
netstat -anpo|grep ssh
tcp 0 0 127.0.0.1:29568 127.0.0.1:22212 ESTABLISHED 18653/ssh keepalive (23.06/0/0)
tcp 0 0 127.0.0.1:29571 127.0.0.1:22212 ESTABLISHED 18726/ssh keepalive (37.48/0/0)
tcp 0 0 127.0.0.1:29570 127.0.0.1:22212 ESTABLISHED 18724/ssh keepalive (18.71/0/0)
tcp 0 0 127.0.0.1:29573 127.0.0.1:22212 ESTABLISHED 18730/ssh keepalive (69.39/0/0)
tcp 0 0 127.0.0.1:29572 127.0.0.1:22212 ESTABLISHED 18728/ssh keepalive (73.43/0/0)
postgres@db-192-1-1-5-> cd $PGDATA
postgres@db-192-1-1-5-> pwd
/data02/pg_root
postgres@db-192-1-1-5-> ll|grep pg_xlog
lrwxrwxrwx 1 postgres postgres 27 Dec 13 18:54 pg_xlog -> /data01/pg_xlog
postgres@db-192-1-1-5-> cd $PGDATA/pg_tblspc
postgres@db-192-1-1-5-> ll
lrwxrwxrwx 1 postgres postgres 24 Feb 20 14:22 16387 -> /data09/digoal
lrwxrwxrwx 1 postgres postgres 31 Mar 6 11:11 16388 -> /data06/tbs_digoal_old
lrwxrwxrwx 1 postgres postgres 28 Dec 13 18:55 16381 -> /data03/tbs_digoal
lrwxrwxrwx 1 postgres postgres 32 Dec 13 18:55 16382 -> /data04/tbs_digoal_idx
lrwxrwxrwx 1 postgres postgres 29 Dec 13 18:56 16389 -> /data05/tbs_digoal1
postgres@db-192-1-1-5-> cat $PGDATA/postgresql.conf|grep log_directory
log_directory = '/var/applog/pg_log' # directory where log files are written,
在standby服务器上创建这些目录:
mkdir -p /data02/pg_root
mkdir -p /data01/pg_xlog
mkdir -p /data09/digoal
mkdir -p /data06/tbs_digoal_old
mkdir -p /data03/tbs_digoal
mkdir -p /data04/tbs_digoal_idx
mkdir -p /data05/tbs_digoal1
mkdir -p /var/applog/pg_log
# 修改为数据库启动用户权限
chown -R postgres:postgres /data02/pg_root
chown -R postgres:postgres /data01/pg_xlog
chown -R postgres:postgres /data09/digoal
chown -R postgres:postgres /data06/tbs_digoal_old
chown -R postgres:postgres /data03/tbs_digoal
chown -R postgres:postgres /data04/tbs_digoal_idx
chown -R postgres:postgres /data05/tbs_digoal1
chown -R postgres:postgres /var/applog/pg_log
# 修改为700
chmod -R 700 /data02/pg_root
chmod -R 700 /data01/pg_xlog
chmod -R 700 /data09/digoal
chmod -R 700 /data06/tbs_digoal_old
chmod -R 700 /data03/tbs_digoal
chmod -R 700 /data04/tbs_digoal_idx
chmod -R 700 /data05/tbs_digoal1
chmod -R 700 /var/applog/pg_log
2. 在standby 节点创建秘钥
postgres@db-192-1-1-1-> ssh-keygen -t rsa
3. 将公钥拷贝到master节点
su - postgres
chmod 700 ~
cd ~/.ssh
vi authorized_keys
#standby节点上~/.ssh/id_rsa.pub内容拷贝到这里
chmod 700 ~/.ssh
chmod 400 ~/.ssh/authorized_keys
# 确保master节点允许公钥验证.
[root@db-192-1-1-5 ~]# cat /etc/ssh/sshd_config |grep Pub
PubkeyAuthentication yes
# 如果如果有超时关闭连接的设备, 建议建立隧道.
ssh -o ServerAliveInterval=90 -o ServerAliveCountMax=60 -o CompressionLevel=9 -p remote_direct_ip_ssh_port -CqTfnN -L *:10022:remote_dest_ip:remote_dest_ssh_port postgres@remote_direct_ip
4. 在standby 节点执行pg_receivexlog (早于pg_start_backup开始, 可以拿到所有需要的pg_xlog文件)
pg_receivexlog -D $PGDATA/pg_xlog -v -h 192.1.1.5 -p 5432 -s 60 -U replica
5. 在master节点执行开始备份
psql
psql (9.2.4)
Type "help" for help.
postgres=# select pg_start_backup(now()::text);
6. master节点执行完pg_start_backup后, 在standby节点执行rsync.
nohup rsync -acvz --progress --rsh='ssh -p 10022' --delete --exclude=pg_xlog* --exclude=*.pid postgres@192.1.1.5:/data02/pg_root/ /data02/pg_root >>/home/postgres/pg_root.log 2>&1 &
nohup rsync -acvz --progress --rsh='ssh -p 10022' --delete postgres@192.1.1.5:/data09/digoal/ /data09/digoal >>/home/postgres/digoal.log 2>&1 &
nohup rsync -acvz --progress --rsh='ssh -p 10022' --delete postgres@192.1.1.5:/data03/tbs_digoal/ /data03/tbs_digoal >>/home/postgres/tbs_digoal.log 2>&1 &
nohup rsync -acvz --progress --rsh='ssh -p 10022' --delete postgres@192.1.1.5:/data05/tbs_digoal1/ /data05/tbs_digoal1 >>/home/postgres/tbs_digoal1.log 2>&1 &
nohup rsync -acvz --progress --rsh='ssh -p 10022' --delete postgres@192.1.1.5:/data04/tbs_digoal_idx/ /data04/tbs_digoal_idx >>/home/postgres/tbs_digoal_idx.log 2>&1 &
nohup rsync -acvz --progress --rsh='ssh -p 10022' --delete postgres@192.1.1.5:/data06/tbs_digoal_old/ /data06/tbs_digoal_old >>/home/postgres/tbs_digoal_old.log 2>&1 &
8. 启动standby
【参考】
8.
pg173212@db-192-168-20-4-> pg_receivexlog --help
pg_receivexlog receives PostgreSQL streaming transaction logs.Usage:pg_receivexlog [OPTION]...Options:-D, --directory=DIR receive transaction log files into this directory-n, --no-loop do not loop on connection lost-v, --verbose output verbose messages-V, --version output version information, then exit-?, --help show this help, then exitConnection options:-h, --host=HOSTNAME database server host or socket directory-p, --port=PORT database server port number-s, --status-interval=INTERVALtime between status packets sent to server (in seconds)-U, --username=NAME connect as specified database user-w, --no-password never prompt for password-W, --password force password prompt (should happen automatically)
Report bugs to <pgsql-bugs@postgresql.org>.
mongoDB VS PostgreSQL dml performance use python (pymongo py-postgresql)相关推荐
- 【Python pymongo】零基础也能轻松掌握的学习路线与参考资料
Python pymongo是一款基于Python的MongoDB数据库的驱动程序,它提供了操作MongoDB数据库的接口和方法.学习Python pymongo可以帮助开发者更好地使用MongoDB ...
- python psycopg2_Ubuntu 安装 PostgreSQL 和 python-psycopg2基础教程(以及错误解决)
Ubuntu 安装 PostgreSQL 和 python-psycopg2基础教程(以及错误解决) 2012/06/27 by Crazyant 3条评论 Django支持以下四种数据库Pos ...
- 再不了解PostgreSQL,你就晚了之PostgreSQL主从流复制部署...
再不了解PostgreSQL,你就晚了之PostgreSQL主从流复制部署 前言 在MySQL被收购之后,虽然有其替代品为: MariaDB,但是总感觉心里有点膈应.大家发现了另一款开源的数据库: P ...
- 解决 win10 pycurl安装出错 Command python setup.py egg_info failed with error code 10 编译安装包 安装万金油...
版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明. 本文链接:https://blog.csdn.net/yexiaohhjk/article/d ...
- Python 运行 Python hello.py 出错,提示: File stdin , line 1
写了一个hello.py,仅有一句,print 'hello world', 运行 Python hello.py 出错,提示: File "<stdin>" , li ...
- Python setup.py开发与安装
本文翻译自:Python setup.py develop vs install Two options in setup.py develop and install are confusing m ...
- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full comm
目录 问题 解决方法 问题 Python3.7环境下安装PySide pip install PySide Collecting PySideUsing cached PySide-1.2.4.tar ...
- python pip 报错 Command “python setup.py egg_info“ failed with error code 1 解决方法
在执行 pip3 install scrapy 时遇到错误: Command "python setup.py egg_info" failed with error code 1 ...
- python setup.py install 安装的包 卸载方法
增加 –record 参数重新安装软件包,执行命令: python ./setup.py install --record install.txt 删除安装文件,执行命令: cat install.t ...
- python导入其他py文件-Python中py文件引用另一个py文件变量的方法
最近自己初学Python,在编程是遇到一个问题就是,怎样在一个py文件中使用另一个py文件中变量,问题如下: demo1代码 import requests r = requests.get(&quo ...
最新文章
- 能量平衡_巴夏:平衡自己的能量,疗愈自己,疗愈世界
- 你想要什么样的财富自由
- 全球十大AI领军人物
- Java实现大数乘法_java实现大数加法、乘法(BigDecimal)
- libusb usb设备访问接口库
- 视频质量评价方法简介
- 每天学点Linux命令:倒叙打印文件第二行的前100个大写字母
- MaxCompute Console 实用小命令
- 走进Windows Server 2008服务器核心(Serve Core)
- linwei_211 SVN错误:Attempted to lock an already-locked dir
- 面试题46. 把数字翻译成字符串
- 无线信号拓展与覆盖的解决方案
- select、autocomplete标签下拉框
- 汇编语言及应用——程序设计
- oracle考试试题及其答案,Oracle期末考试试题及答案-
- web前端课程设计:猫咪领养网站 HTML+CSS+JavaScript
- OpenGL日常-GLAD
- Android的序列化(Serializable和Parcelable)
- kaggle——泰坦尼克数据集
- wazuh官方安装指南(中文译版本)