前言

上次分享了barman备份的安装部署过程，这次将继续分享一下barman备份恢复工具的使用过程。
barman是目前postgreSQL数据库的比较成熟的一款备份工具，是基于postgresql的一款物理备份工具。与oracle数据库中的rman较为类似。
其简单的工作原理即为：
物理全备+归档备份（WAL日志备份），从而达到任意时间点的恢复。即恢复全备后，通过指定WAL日志的时间点、LSN等即可恢复到数据库的任意时间点。
barman备份主要分为三类：
1.仅流复制；
2.ssh/rsync;
3.流复制+ssh/rsync

上一次，已经完成了barman工具的安装部署工作。本文中，将对postgresql和barman进行配置并测试备份恢复。

一、postgreSQL配置

参数配置（具体参数可以根据实际情况配置）：

listen_addresses = '*'
port = 18801
max_connections = 100
superuser_reserved_connections = 3
tcp_keepalives_idle = 7200
tcp_keepalives_interval = 130
tcp_keepalives_count = 3
shared_buffers = 256MB
work_mem = 16MB
maintenance_work_mem = 128MB
wal_level = logical
full_page_writes = on
max_wal_size = 1GB
min_wal_size = 80MB
wal_keep_segments = 32
logging_collector = on
log_directory = '/PgData/pg_log'
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
log_truncate_on_rotation = on
log_rotation_age = 1d
log_rotation_size = 0
log_min_messages = info
log_min_error_statement = infoshared_preload_libraries = 'pg_stat_statements' #加载pg_stat_statements模块
track_io_timing = on              #跟踪IO消耗的时间
pg_stat_statements.max = 10000    #最多保留多少条统计信息，通过LRU算法，覆盖老的记录。
pg_stat_statements.track = all    #all：所有SQL包括函数内嵌套的SQL,top：直接执行的SQL(函数内的sql不被跟踪),none：不跟踪
pg_stat_statements.track_utility=on # 是否跟踪非DML语句 (例如DDL，DCL)，on表示跟踪, off表示不跟踪
pg_stat_statements.save = on      # 重启后是否保留统计信息
#track_activity_query_size = 2048  #设置单条SQL的最长长度，超过被截断显示

修改pg_hba.conf

host    all             barman          0.0.0.0/0               trust
host    replication     barmstr       0.0.0.0/0       trust

postgresql数据库上创建用户（barman需要superuser权限，barmanStr需要replication权限）：

create user barman superuser password 'barman';
create user barmanStr replication password 'barman';

重启数据库：

pg_ctl -D /PgData start

连接尝试：

在备份服务器上尝试连接验证：

psql -h 192.168.138.141 -U barman -d wangxin -p 18801
psql -h 192.168.138.141 -U barmstr -d wangxin -p 18801

二、barman配置：

创建用户：

useradd barman

修改barman的配置文件：

vi /etc/barman.conf

[barman]
barman_user = barman
configuration_files_directory = /etc/barman.d
;指定barman_home，即备份文件的存放路径
barman_home = /home/barman
log_file = /home/barman/barman.log
log_level = INFO
compression = gzip
parallel_jobs = 1
immediate_checkpoint = true
basebackup_retry_times = 3
basebackup_retry_sleep = 60
check_timeout = 120
retention_policy = REDUNDANCY 2
retention_policy = RECOVERY WINDOW OF 4 WEEKS

修改服务配置文件：

cp streaming-server.conf-template pgsql-streaming.conf
vi pgsql-streaming.conf

内容如下：

[pgsql_streaming]
description =  "pgsql11-streaming (Streaming-Only)"
conninfo = host=pgsql11 user=barman dbname=wangxin port=18801
streaming_conninfo = host=pgsql11 user=barmanstr port=18801
backup_method = postgres
streaming_backup_name = barman_streaming_backup
streaming_archiver = on
archiver = on
slot_name = pgsql_streaming
create_slot = auto
streaming_archiver_name = barman_receive_wal
streaming_archiver_batch_size = 50
path_prefix = "/pg_client/bin"

配置服务端和备份端的ssh免密登录：
pgsql:

ssh-keygen -t rsacat id_rsa.pub >> authorized_keysscp authorized_keys >> barman@barman:/home/barman/.ssh/

barman:

ssh-keygen -t rsacat id_rsa.pub >> authorized_keysscp authorized_keys >> pgsql@pgsql11:/home/pgsql/.ssh/

pgsql:

chmod -R 600 /home/pgsql/.ssh/authorized_keys

barman:

chmod -R 600 /home/barman/.ssh/authorized_keys

创建slot槽：

barman receive-wal --create-slot pgsql_streaming

进行check检查：

barman check pgsql_streaming

查看连接服务的信息

[barman@barman ~]$ barman show-server pgsql_streaming
Server pgsql_streaming:active: Truearchive_command: (disabled)archive_mode: offarchive_timeout: 0archived_count: 0archiver: Truearchiver_batch_size: 0backup_directory: /home/barman/pgsql_streamingbackup_method: postgresbackup_options: BackupOptions({'concurrent_backup'})bandwidth_limit: Nonebarman_home: /home/barmanbarman_lock_directory: /home/barmanbasebackup_retry_sleep: 60basebackup_retry_times: 3basebackups_directory: /home/barman/pgsql_streaming/basecheck_timeout: 120checkpoint_timeout: 300compression: gzipconfig_file: /PgData/postgresql.confconnection_error: Noneconninfo: host=pgsql11 user=barman dbname=wangxin port=18801create_slot: autocurrent_archived_wals_per_second: 0.0current_lsn: 0/669C3780current_size: 1134777182current_xlog: 000000010000000000000066custom_compression_filter: Nonecustom_decompression_filter: Nonedata_checksums: offdata_directory: /PgDatadescription: pgsql11-streaming (Streaming-Only)disabled: Falseerrors_directory: /home/barman/pgsql_streaming/errorsfailed_count: 0has_backup_privileges: Truehba_file: /PgData/pg_hba.confhot_standby: onident_file: /PgData/pg_ident.confimmediate_checkpoint: True**incoming_wals_directory: /home/barman/pgsql_streaming/incoming**is_archiving: Falseis_in_recovery: Falseis_superuser: Truelast_archived_time: Nonelast_archived_wal: Nonelast_backup_maximum_age: Nonelast_failed_time: Nonelast_failed_wal: Nonemax_incoming_wals_queue: Nonemax_replication_slots: 10max_wal_senders: 10minimum_redundancy: 0msg_list: []name: pgsql_streamingnetwork_compression: Falseparallel_jobs: 1passive_node: Falsepath_prefix: /pg_client/binpg_basebackup_bwlimit: Truepg_basebackup_compatible: Truepg_basebackup_installed: Truepg_basebackup_path: /pg_client/bin/pg_basebackuppg_basebackup_tbls_mapping: Truepg_basebackup_version: 11.4pg_receivexlog_compatible: Truepg_receivexlog_installed: Truepg_receivexlog_path: /pg_client/bin/pg_receivewalpg_receivexlog_supports_slots: Truepg_receivexlog_synchronous: Falsepg_receivexlog_version: 11.4pgespresso_installed: Falsepost_archive_retry_script: Nonepost_archive_script: Nonepost_backup_retry_script: Nonepost_backup_script: Nonepost_delete_retry_script: Nonepost_delete_script: Nonepost_recovery_retry_script: Nonepost_recovery_script: Nonepost_wal_delete_retry_script: Nonepost_wal_delete_script: Nonepostgres_systemid: 6901481653023568481pre_archive_retry_script: Nonepre_archive_script: Nonepre_backup_retry_script: Nonepre_backup_script: Nonepre_delete_retry_script: Nonepre_delete_script: Nonepre_recovery_retry_script: Nonepre_recovery_script: Nonepre_wal_delete_retry_script: Nonepre_wal_delete_script: Noneprimary_ssh_command: Nonerecovery_options: RecoveryOptions()replication_slot: Record(slot_name='pgsql_streaming', active=False, restart_lsn=None)replication_slot_support: Trueretention_policy: RECOVERY WINDOW OF 4 WEEKSretention_policy_mode: autoreuse_backup: Noneserver_txt_version: 11.4slot_name: pgsql_streamingssh_command: Nonestats_reset: 2020-12-02 10:33:56.983254+08:00streaming: Truestreaming_archiver: Truestreaming_archiver_batch_size: 50streaming_archiver_name: barman_receive_walstreaming_backup_name: barman_streaming_backupstreaming_conninfo: host=pgsql11 user=barmanstr port=18801streaming_supported: Truestreaming_systemid: 6901481653023568481streaming_wals_directory: /home/barman/pgsql_streaming/streamingsynchronous_standby_names: ['']tablespace_bandwidth_limit: Nonetimeline: 1wal_compression: offwal_keep_segments: 0wal_level: logicalwal_retention_policy: MAINwals_directory: /home/barman/pgsql_streaming/walsxlog_segment_size: 16777216xlogpos: 0/669C3780

检查barman是否满足备份要求：

[barman@barman ~]$ barman check pgsql_streaming
Server pgsql_streaming:WAL archive: FAILED (please make sure WAL shipping is setup)PostgreSQL: OKsuperuser or standard user with backup privileges: OKPostgreSQL streaming: OKwal_level: OKreplication slot: FAILED (slot 'pgsql_streaming' not initialised: is 'receive-wal' running?)directories: OKretention policy settings: OKbackup maximum age: OK (no last_backup_maximum_age provided)compression settings: OKfailed backups: OK (there are 0 failed backups)minimum redundancy requirements: OK (have 0 backups, expected at least 0)pg_basebackup: OKpg_basebackup compatible: OKpg_basebackup supports tablespaces mapping: OKsystemid coherence: OK (no system Id stored on disk)pg_receivexlog: OKpg_receivexlog compatible: OKreceive-wal running: FAILED (See the Barman log file for more details)archive_mode: FAILED (please set it to 'on' or 'always')archive_command: FAILED (please set it accordingly to documentation)archiver errors: OK

可以关注到一个错误：
replication slot: FAILED (slot ‘pgsql_streaming’ not initialised: is ‘receive-wal’ running?)

解决该错误，我们需要执行以下命令，让barman服务器来强制接受源端的wal日志：

nohup barman receive-wal pgsql_streaming &

再次进行check：

[barman@barman ~]$ barman check pgsql_streaming
Server pgsql_streaming:WAL archive: FAILED (please make sure WAL shipping is setup)PostgreSQL: OKsuperuser or standard user with backup privileges: OKPostgreSQL streaming: OKwal_level: OKreplication slot: OKdirectories: OKretention policy settings: OKbackup maximum age: OK (no last_backup_maximum_age provided)compression settings: OKfailed backups: OK (there are 0 failed backups)minimum redundancy requirements: OK (have 0 backups, expected at least 0)pg_basebackup: OKpg_basebackup compatible: OKpg_basebackup supports tablespaces mapping: OKsystemid coherence: OK (no system Id stored on disk)pg_receivexlog: OKpg_receivexlog compatible: OKreceive-wal running: OKarchive_mode: FAILED (please set it to 'on' or 'always')archive_command: FAILED (please set it accordingly to documentation)archiver errors: OK

此时，可以发现，错误只有以下内容：
WAL archive: FAILED (please make sure WAL shipping is setup)
archive_mode: FAILED (please set it to ‘on’ or ‘always’)
archive_command: FAILED (please set it accordingly to documentation)

但是这三个错误是通过ssh/rsync的方式，如果我们只用streaming的方式可以忽略，然后直接发起备份，当然如果想要处理这些问题，可以这样做：

1.问题一

执行barman cron:Starting WAL archiving for server pgsql_streaming执行barman switch-xlogbarman switch-xlog pgsql_streaming

2.问题二

根据提示在server端设置archive_mod和archive_command参数：vi postgresql.confarchive_mod = on
archive_command = 'rsync -a %p barman@barman:/home/barman/pgsql_streaming/incoming/%f' #该路径就是我们刚在barman show-server pg_streaming中查询的

重启数据库
此时再次执行check可以看到状态都正常了：
[root@barman barman.d]# barman check pgsql_streaming
Server pgsql_streaming:
PostgreSQL: OK
superuser or standard user with backup privileges: OK
PostgreSQL streaming: OK
wal_level: OK
replication slot: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
pg_basebackup: OK
pg_basebackup compatible: OK
pg_basebackup supports tablespaces mapping: OK
systemid coherence: OK (no system Id stored on disk)
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archive_mode: OK
archive_command: OK
continuous archiving: OK
archiver errors: OK

全部正常以后，我们则可以开始进行备份。

三、barman备份：
进行备份：

barman backup pgsql_streaming

查看备份：

barman list-backup pgsql_streaming

查看备份详细信息

barman show-backup pgsql_streaming 20201204T164310Backup 20201204T164310:Server Name            : pgsql_streamingSystem Id              : 6901481653023568481Status                 : DONEPostgreSQL Version     : 110004PGDATA directory       : /PgDataBase backup information:Disk usage           : 1.1 GiB (1.1 GiB with WALs)Incremental size     : 1.1 GiB (-0.00%)Timeline             : 1Begin WAL            : 00000001000000000000006FEnd WAL              : 00000001000000000000006FWAL number           : 1WAL compression ratio: 99.90%Begin time           : 2020-12-04 16:43:10+08:00End time             : 2020-12-04 16:43:39.799061+08:00Copy time            : 29 secondsEstimated throughput : 37.4 MiB/sBegin Offset         : 40End Offset           : 0Begin LSN           : 0/6F000028End LSN             : 0/70000000WAL information:No of files          : 0Disk usage           : 0 BLast available       : 00000001000000000000006FCatalog information:Retention Policy     : VALIDPrevious Backup      : - (this is the oldest base backup)Next Backup          : - (this is the latest base backup)

实时查看备份情况:

[root@barman .ssh]# barman status pgsql_streaming
Server pgsql_streaming:Description: pgsql11-streaming (Streaming-Only)Active: TrueDisabled: FalsePostgreSQL version: 11.4Cluster state: in productionpgespresso extension: Not availableCurrent data size: 1.1 GiBPostgreSQL Data directory: /PgDataCurrent WAL segment: 000000010000000000000073PostgreSQL 'archive_command' setting: rsync -a %p barman@barman:/home/barman/pgsql_streaming/incoming/%fLast archived WAL: 000000010000000000000072, at Fri Dec  4 16:50:13 2020Failures of WAL archiver: 0Server WAL archiving rate: 0.26/hourPassive node: FalseRetention policies: enforced (mode: auto, retention: RECOVERY WINDOW OF 4 WEEKS, WAL retention: MAIN)No. of available backups: 1First available backup: 20201204T164310Last available backup: 20201204T164310Minimum redundancy requirements: satisfied (1/0)

说明：
配置文件中的barman_home，即为备份存放路径
打开barman_home下对应的备份服务下，可以看到以下目录：
drwxrwxr-x. 2 barman barman 6 Dec 4 15:36 errors
-rw-r–r--. 1 barman barman 63 Dec 4 16:43 identity.json
drwxrwxr-x. 3 barman barman 45 Dec 4 16:43 wals
drwxrwxr-x. 5 barman barman 75 Dec 4 17:14 base
drwxrwxr-x. 2 barman barman 78 Dec 4 17:14 streaming
drwxrwxr-x. 2 barman barman 6 Dec 4 17:14 incoming
其中，比较重要的有wals、base、streaming、incoming

base中，存放的是数据库的基本备份文件(最近的一份全备)
streaming中，存放的是实时传过来的wal文件
incoming中，存放的是主库传输过来的归档文件
wals中，是所有的归档文件，可以根据这个和基础备份恢复到任意时间点

barman恢复：

查看拥有的备份集：

barman list-backup pgsql_streaming
pgsql_streaming 20201207T231544 - Sun Dec  6 23:15:56 2020 - Size: 356.9 MiB - WAL Size: 0 B
pgsql_streaming 20201207T225457 - Sun Dec  6 22:55:44 2020 - Size: 1.1 GiB - WAL Size: 4.4 MiB
pgsql_streaming 20201204T171433 - Fri Dec  4 17:14:42 2020 - Size: 1.1 GiB - WAL Size: 6.3 MiB
pgsql_streaming 20201204T170853 - Fri Dec  4 17:09:11 2020 - Size: 1.1 GiB - WAL Size: 503.1 KiB
pgsql_streaming 20201204T164310 - Fri Dec  4 16:43:39 2020 - Size: 1.1 GiB - WAL Size: 2.6 MiB

检查选定的备份的详细信息:

[barman@barman 0000000100000000]$ barman show-backup pgsql_streaming 20201207T225457
Backup 20201207T225457:Server Name            : pgsql_streamingSystem Id              : 6901481653023568481Status                 : DONEPostgreSQL Version     : 110004PGDATA directory       : /PgDataBase backup information:Disk usage           : 1.1 GiB (1.1 GiB with WALs)Incremental size     : 1.1 GiB (-0.00%)Timeline             : 1Begin WAL            : 00000001000000000000007CEnd WAL              : 00000001000000000000007EWAL number           : 3WAL compression ratio: 99.90%Begin time           : 2020-12-06 22:55:02+08:00End time             : 2020-12-06 22:55:44.674452+08:00Copy time            : 42 secondsEstimated throughput : 26.4 MiB/sBegin Offset         : 40End Offset           : 96Begin LSN           : 0/7C000028End LSN             : 0/7E000060WAL information:No of files          : 4Disk usage           : 4.4 MiBCompression ratio    : 93.14%Last available       : 000000010000000000000082Catalog information:Retention Policy     : VALIDPrevious Backup      : 20201204T171433Next Backup          : 20201207T231544

barman恢复（恢复到远端数据库）：

[barman@barman PgData_recover]$ barman recover pgsql_streaming 20201207T225457 /PgData --remote-ssh-command "ssh postgres@pgsql11-recover" --target-time "2020-12-06 22:57:00.0000"
Processing xlog segments from streaming for pgsql_streaming000000010000000000000083
Processing xlog segments from file archival for pgsql_streaming000000010000000000000083
The authenticity of host 'pgsql11-recover (192.168.138.143)' can't be established.
ECDSA key fingerprint is SHA256:vu/gUNno3I8feQxKWcPkcm2kSS+g72zAE2JafM9CBF4.
ECDSA key fingerprint is MD5:59:a7:b9:12:29:31:b3:78:7f:ed:71:44:7a:7e:f2:fc.
Are you sure you want to continue connecting (yes/no)? yes
postgres@pgsql11-recover's password:
Starting remote restore for server pgsql_streaming using backup 20201207T225457
Destination directory: /PgData
Remote command: ssh postgres@pgsql11-recover
Doing PITR. Recovery target time: '2020-12-06 22:57:00+08:00'
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
Copying the base backup.
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
Copying required WAL segments.
postgres@pgsql11-recover's password:
Generating recovery configuration
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
postgres@pgsql11-recover's password:
Identify dangerous settings in destination directory.
postgres@pgsql11-recover's password: IMPORTANT
These settings have been modified to prevent data lossespostgresql.conf line 706: archive_command = falseRecovery completed (start time: 2020-12-07 23:57:04.056352, elapsed time: 1 minute, 54 seconds)Your PostgreSQL server has been successfully prepared for recovery!

出现：
Your PostgreSQL server has been successfully prepared for recovery!
则表示恢复成功。

通过pg_dump/dblink等方式，将恢复的表或者数据回写到源端。

barman备份的配置使用（备份+恢复）相关推荐

xtrabackup备份mysql_配置xtrabackup备份mysql数据库
下载地址: https://www.percona.com/downloads/XtraBackup/LATEST/ 为了方便起见本次安装使用yum源安装方式 1 安装yum源: yum ins ...
升级mysql服务器二进制备份_MySQL二进制日志备份和恢复详解
基本概念定义: 二进制日志包含了所有更新了数据或者已经潜在更新了数据(例如,没有匹配任何行的一个DELETE)的所有语句. 作用: 1.二进制日志的主要目的是在恢复使能够最大可能地更新数据库,因为二 ...
mysql 物理备份工具_mysql物理备份工具Xtrabackup安装配置
mysql物理备份工具Xtrabackup安装配置 1Xtrabackup工具介绍 Xtrabackup是一个对InnoDB做物理数据备份的工具,支持在线热备份(备份时不影响数据读写),是商业备份工具 ...
db2设置数据库增量备份_DB2在线增量备份还原增量备份及前滚恢复
1.更改数据库参数 logretain, userexit, trackmod 为 on 2.更改参数之后完全离线备份数据库一次 3.之后就可以进行在线.在线增量备份了测试结果通过,脚本如下. 重要 ...
PostgreSQL 最佳实践 - 在线增量备份与任意时间点恢复
背景冷备份, 以及逻辑备份都是某一个时间点的备份, 没有增量的概念. 如果数据库在运行过程中发生故障, 使用逻辑备份只能将数据库还原到备份时刻, 无法恢复到故障发生前的那个时刻. 又或者在使用过程中 ...
VI3之vCenterServer配置的备份与还原
在Vmware Infrastruacture 3中,vCenter Server起到非常重要的角色.在VMware给出的一些解决方案中,常见的是针对Esx主机高用性的VMware HA等的解决方案, ...
dlink 备份文件_D-Link路由器备份路由器配置信息教程
作为不同网络之间互相连接的枢纽,路由器系统构成了基于TCP/IP 的国际互联网络Internet 的主体脉络,也可以说,路由器构成了Internet的骨架.一些路由器经常死机,很多时候需要重置路由器, ...
制品仓库 Nexus 安装、配置、备份、使用
目录 1.1 Nexus 优点 1.2 Nexus 仓库类型 2. 安装 Nexus 2.1 设置持久化目录 2.2 拉取 Nexus docker 镜像 2.3 运行并启动 Nexus 3. 系统配 ...
mysql 查看备份工具_MariaDB/MySQL备份和恢复(一)：mysqldump工具用法详述【转】
本人博客搬家(本站无精力更新和维护):骏马金龙www.junmajinlong.com 1.备份分类按照是否能够继续提供服务,将数据库备份类型划分为: 热备份:在线备份,能读能写温备份:能读不能写 ...
oracle配置并发备份,oracle数据库还原以及备份包括快速备份（并发压缩）
expdp jhpt/XXXX directory=databackup dumpfile=dpfile_201512091300_%U.dmp filesize=5G parallel=8 comp ...

barman备份的配置使用（备份+恢复）

前言