Greenplum 表空间和filespace的用法
Greenplum支持表空间,创建表空间时,需要指定filespace。
postgres=# \h create tablespace;
Command: CREATE TABLESPACE
Description: define a new tablespace
Syntax:
CREATE TABLESPACE tablespace_name [OWNER username] FILESPACE filespace_name那么什么是filespace呢?
GP在初始化完后,有一个默认的filespace : pg_system。
表空间pg_default和pg_global都放在这个filespace下面。
也就是说一个filespace可以被多个表空间使用。
postgres=# select oid,* from pg_filespace;oid | fsname | fsowner
------+-----------+---------3052 | pg_system | 10
(1 row)
postgres=# select * from pg_tablespace;spcname | spcowner | spclocation | spcacl | spcprilocations | spcmirlocations | spcfsoid
------------+----------+-------------+--------+-----------------+-----------------+----------pg_default | 10 | | | | | 3052pg_global | 10 | | | | | 3052
(2 rows)还有TEMPORARY_FILES和TRANSACTION_FILES对应的filespace如下:
$gpfilespace --showtempfilespace
20151218:16:02:07:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-
A tablespace requires a file system location to store its database
files. A filespace is a collection of file system locations for all components
in a Greenplum system (primary segment, mirror segment and master instances).
Once a filespace is created, it can be used by one or more tablespaces.20151218:16:02:07:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-Getting filespace information for TEMPORARY_FILES
20151218:16:02:08:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-Checking for filespace consistency
20151218:16:02:08:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-Obtaining current filespace entries used by TEMPORARY_FILES
20151218:16:02:09:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-TEMPORARY_FILES OIDs are consistent for pg_system filespace
20151218:16:02:11:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-TEMPORARY_FILES entries are consistent for pg_system filespace
20151218:16:02:11:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-Obtaining current filespace entries used by TEMPORARY_FILES
20151218:16:02:11:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-Current Filespace for TEMPORARY_FILES is pg_system
20151218:16:02:11:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-1 /disk1/digoal/gpdata/gpseg-1
20151218:16:02:11:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-25 /disk1/digoal/gpdata_mirror/gpseg0
20151218:16:02:11:063949 gpfilespace:127.0.0.1:digoal-[INFO]:-2 /disk1/digoal/gpdata/gpseg0
......$gpfilespace --showtransfilespace
20151218:16:09:41:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-
A tablespace requires a file system location to store its database
files. A filespace is a collection of file system locations for all components
in a Greenplum system (primary segment, mirror segment and master instances).
Once a filespace is created, it can be used by one or more tablespaces.20151218:16:09:41:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-Getting filespace information for TRANSACTION_FILES
20151218:16:09:41:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-Checking for filespace consistency
20151218:16:09:41:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-Obtaining current filespace entries used by TRANSACTION_FILES
20151218:16:09:42:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-TRANSACTION_FILES OIDs are consistent for pg_system filespace
20151218:16:09:44:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-TRANSACTION_FILES entries are consistent for pg_system filespace
20151218:16:09:44:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-Obtaining current filespace entries used by TRANSACTION_FILES
20151218:16:09:44:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-Current Filespace for TRANSACTION_FILES is pg_system
20151218:16:09:44:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-1 /disk1/digoal/gpdata/gpseg-1
20151218:16:09:44:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-25 /disk1/digoal/gpdata_mirror/gpseg0
20151218:16:09:44:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-2 /disk1/digoal/gpdata/gpseg0
20151218:16:09:44:071104 gpfilespace:127.0.0.1:digoal-[INFO]:-26 /disk1/digoal/gpdata_mirror/gpseg1
......如果我们的greenplum集群中,有SSD硬盘,又有SATA硬盘。怎样更好的利用这些空间呢?
方法1.
用flashcache或bcache,通过device mapper技术,将SSD和SATA绑定,做成块设备。
再通过 逻辑卷管理 或者 软RAID 或者 brtfs or zfs管理起来,做成大的文件系统。
(还有一种方法是用RHEL 7提供的LVM,可以将SSD作为二级缓存)
这种方法对GP来说,是混合动力,可以创建一个或多个文件系统(都具备混合动力)。
所以建议只需要一个pg_system filespace就够了(除非容量到了文件系统管理的极限,那样的话可以分成多个文件系统)。
用多个文件系统的情况下,就需要对每个文件系统,创建对应的目录,以及filespace。方法2.
SSD和SATA分开,各自创建各自的文件系统。
对每个文件系统,创建对应的目录,以及filespace。创建filespace非常简单,分几步。
如下:
1. 创建目录,需要在所有的角色对应的主机中创建。给予gp 操作系统管理用户对应的权限。
master
$ mkdir /ssd1/gpdata/master_p
$ chown gpadmin:gpadmin /ssd1/gpdata/master_pmaster standby
$ mkdir /ssd1/gpdata/master_s
$ chown gpadmin:gpadmin /ssd1/gpdata/master_ssegment
$ mkdir /ssd1/gpdata_p
$ chown gpadmin:gpadmin /ssd1/gpdata_psegment mirror
$ mkdir /ssd1/gpdata_s
$ chown gpadmin:gpadmin /ssd1/gpdata_s2. 查看系统配置
postgres=# select dbid,content,role,preferred_role,hostname,port from gp_segment_configuration order by role,dbid;dbid | content | role | preferred_role | hostname | port
------+---------+------+----------------+-------------------+-------2 | 0 | m | p | digoal.sqa.zmf | 400003 | 1 | m | p | digoal.sqa.zmf | 400014 | 2 | m | p | digoal.sqa.zmf | 400025 | 3 | m | p | digoal.sqa.zmf | 400036 | 4 | m | p | digoal.sqa.zmf | 400047 | 5 | m | p | digoal.sqa.zmf | 400058 | 6 | m | p | digoal.sqa.zmf | 400069 | 7 | m | p | digoal.sqa.zmf | 4000710 | 8 | m | p | digoal.sqa.zmf | 4000811 | 9 | m | p | digoal.sqa.zmf | 4000912 | 10 | m | p | digoal.sqa.zmf | 4001013 | 11 | m | p | digoal.sqa.zmf | 4001114 | 12 | m | p | digoal.sqa.zmf | 4001215 | 13 | m | p | digoal.sqa.zmf | 4001316 | 14 | m | p | digoal.sqa.zmf | 4001417 | 15 | m | p | digoal.sqa.zmf | 4001518 | 16 | m | p | digoal.sqa.zmf | 4001619 | 17 | m | p | digoal.sqa.zmf | 4001720 | 18 | m | p | digoal.sqa.zmf | 4001821 | 19 | m | p | digoal.sqa.zmf | 4001922 | 20 | m | p | digoal.sqa.zmf | 4002023 | 21 | m | p | digoal.sqa.zmf | 4002124 | 22 | m | p | digoal.sqa.zmf | 500111 | -1 | p | p | digoal.sqa.zmf | 192125 | 0 | p | m | digoal.sqa.zmf | 4100026 | 1 | p | m | digoal.sqa.zmf | 4100127 | 2 | p | m | digoal.sqa.zmf | 4100228 | 3 | p | m | digoal.sqa.zmf | 4100329 | 4 | p | m | digoal.sqa.zmf | 4100430 | 5 | p | m | digoal.sqa.zmf | 4100531 | 6 | p | m | digoal.sqa.zmf | 4100632 | 7 | p | m | digoal.sqa.zmf | 4100733 | 8 | p | m | digoal.sqa.zmf | 4100834 | 9 | p | m | digoal.sqa.zmf | 4100935 | 10 | p | m | digoal.sqa.zmf | 4101036 | 11 | p | m | digoal.sqa.zmf | 4101137 | 12 | p | m | digoal.sqa.zmf | 4101238 | 13 | p | m | digoal.sqa.zmf | 4101339 | 14 | p | m | digoal.sqa.zmf | 4101440 | 15 | p | m | digoal.sqa.zmf | 4101541 | 16 | p | m | digoal.sqa.zmf | 4101642 | 17 | p | m | digoal.sqa.zmf | 4101743 | 18 | p | m | digoal.sqa.zmf | 4101844 | 19 | p | m | digoal.sqa.zmf | 4101945 | 20 | p | m | digoal.sqa.zmf | 4102046 | 21 | p | m | digoal.sqa.zmf | 4102147 | 22 | p | m | digoal.sqa.zmf | 41022
(47 rows)3. 创建配置文件,格式如下,假如我要创建一个名为ssd1的filespace。
字段包含(hostname, dbid, DIR/$prefix + $content)
$ vi conf
filespace:ssd1
digoal.sqa.zmf:1:/ssd1/gpdata/master_p/gp-1
digoal.sqa.zmf:2:/ssd1/gpdata_p/gp0
digoal.sqa.zmf:3:/ssd1/gpdata_p/gp1
......
digoal.sqa.zmf:25:/ssd1/gpdata_s/gp0
digoal.sqa.zmf:26:/ssd1/gpdata_s/gp1
......还有一种方法是使用gpfilespace -o conf来生成配置文件。(在提示时输入目录名DIR的部分即可)4. 创建filespace
gpfilespace -c conf -h 127.0.0.1 -p 1921 -U digoal -W digoal20151218:17:16:39:108364 gpfilespace:127.0.0.1:digoal-[INFO]:-Connecting to database
20151218:17:16:39:108364 gpfilespace:127.0.0.1:digoal-[INFO]:-Filespace "ssd1" successfully created
......
然后gpfilespace会自动在数据库执行以下DDL SQL。创建对应的filespace。
所以我们也可以自己在数据库中执行SQL来创建filespace。CREATE FILESPACE ssd1
(1: '/disk1/digoal/new_p/gp-1',2: '/disk1/digoal/new_p/gp0',3: '/disk1/digoal/new_p/gp1',4: '/disk1/digoal/new_p/gp2',5: '/disk1/digoal/new_p/gp3',6: '/disk1/digoal/new_p/gp4',7: '/disk1/digoal/new_p/gp5',8: '/disk1/digoal/new_p/gp6',9: '/disk1/digoal/new_p/gp7',10: '/disk1/digoal/new_p/gp8',11: '/disk1/digoal/new_p/gp9',12: '/disk1/digoal/new_p/gp10',13: '/disk1/digoal/new_p/gp11',14: '/disk1/digoal/new_p/gp12',15: '/disk1/digoal/new_p/gp13',16: '/disk1/digoal/new_p/gp14',17: '/disk1/digoal/new_p/gp15',18: '/disk1/digoal/new_p/gp16',19: '/disk1/digoal/new_p/gp17',20: '/disk1/digoal/new_p/gp18',21: '/disk1/digoal/new_p/gp19',22: '/disk1/digoal/new_p/gp20',23: '/disk1/digoal/new_p/gp21',24: '/disk1/digoal/new_p/gp22',25: '/disk1/digoal/new_s/gp0',26: '/disk1/digoal/new_s/gp1',27: '/disk1/digoal/new_s/gp2',28: '/disk1/digoal/new_s/gp3',29: '/disk1/digoal/new_s/gp4',30: '/disk1/digoal/new_s/gp5',31: '/disk1/digoal/new_s/gp6',32: '/disk1/digoal/new_s/gp7',33: '/disk1/digoal/new_s/gp8',34: '/disk1/digoal/new_s/gp9',35: '/disk1/digoal/new_s/gp10',36: '/disk1/digoal/new_s/gp11',37: '/disk1/digoal/new_s/gp12',38: '/disk1/digoal/new_s/gp13',39: '/disk1/digoal/new_s/gp14',40: '/disk1/digoal/new_s/gp15',41: '/disk1/digoal/new_s/gp16',42: '/disk1/digoal/new_s/gp17',43: '/disk1/digoal/new_s/gp18',44: '/disk1/digoal/new_s/gp19',45: '/disk1/digoal/new_s/gp20',46: '/disk1/digoal/new_s/gp21',47: '/disk1/digoal/new_s/gp22'
);现在你可以使用这个filespace了.
例如
1. 将temp , trans移动到这个新的filespace.
$gpfilespace --movetempfilespace ssd120151218:17:17:29:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-
A tablespace requires a file system location to store its database
files. A filespace is a collection of file system locations for all components
in a Greenplum system (primary segment, mirror segment and master instances).
Once a filespace is created, it can be used by one or more tablespaces.20151218:17:17:29:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Database was started in NORMAL mode
20151218:17:17:29:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Stopping Greenplum Database
20151218:17:17:57:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Starting Greenplum Database in master only mode
20151218:17:18:02:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Checking if filespace ssd1 exists
20151218:17:18:02:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Checking if filespace is same as current filespace
20151218:17:18:02:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Stopping Greenplum Database in master only mode
20151218:17:18:04:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Checking for connectivity
20151218:17:18:04:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Obtaining current filespace information
20151218:17:18:04:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Obtaining current filespace entries used by TEMPORARY_FILES
20151218:17:18:04:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Obtaining segment information ...
20151218:17:18:04:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Creating RemoteOperations list
20151218:17:18:04:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Moving TEMPORARY_FILES filespace from pg_system to ssd1 ...
20151218:17:18:06:008363 gpfilespace:127.0.0.1:digoal-[INFO]:-Starting Greenplum Database$gpfilespace --movetransfilespace ssd1
...
20151218:17:19:17:055389 gpfilespace:127.0.0.1:digoal-[INFO]:-Moving TRANSACTION_FILES filespace from pg_system to ssd1 ...
20151218:17:21:16:055389 gpfilespace:127.0.0.1:digoal-[INFO]:-Starting Greenplum Database2. 创建表空间,使用这个filespace.
postgres=# create tablespace tbs_ssd1 filespace ssd1;
CREATE TABLESPACE
postgres=# create table tt(id int) tablespace tbs_ssd1;
NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'id' as the Greenplum Database data distribution key for this table.
HINT: The 'DISTRIBUTED BY' clause determines the distribution of data. Make sure column(s) chosen are the optimal data distribution key to minimize skew.
CREATE TABLE
postgres=# select * from pg_tablespace ;spcname | spcowner | spclocation | spcacl | spcprilocations | spcmirlocations | spcfsoid
------------+----------+-------------+--------+-----------------+-----------------+----------pg_default | 10 | | | | | 3052pg_global | 10 | | | | | 3052tbs_ssd1 | 10 | | | | | 69681
(3 rows)postgres=# select * from pg_filespace;fsname | fsowner
-----------+---------pg_system | 10ssd1 | 10
(2 rows)greenplum为什么会引入filespace的概念?
因为主机目录结构可能不一样,所以原有的目录结构式的方法来创建表空间,可能不够灵活。最后,如何查看每个节点的filespace和location的关系?
digoal=# select a.dbid,a.content,a.role,a.port,a.hostname,b.fsname,c.fselocation from gp_segment_configuration a,pg_filespace b,pg_filespace_entry c where a.dbid=c.fsedbid and b.oid=c.fsefsoid order by content;dbid | content | role | port | hostname | fsname | fselocation
------+---------+------+-------+-------------------+-----------+--------------------------------------1 | -1 | p | 1921 | digoal193096.zmf | pg_system | /data01/gpdata/master_pgdata/gpseg-12 | 0 | p | 40000 | digoal193096.zmf | pg_system | /data01/gpdata/gpseg03 | 1 | p | 40001 | digoal193096.zmf | pg_system | /data01/gpdata/gpseg14 | 2 | p | 40002 | digoal193096.zmf | pg_system | /data01/gpdata/gpseg25 | 3 | p | 40000 | digoal199092.zmf | pg_system | /data01/gpdata/gpseg36 | 4 | p | 40001 | digoal199092.zmf | pg_system | /data01/gpdata/gpseg47 | 5 | p | 40002 | digoal199092.zmf | pg_system | /data01/gpdata/gpseg58 | 6 | p | 40000 | digoal200164.zmf | pg_system | /data01/gpdata/gpseg69 | 7 | p | 40001 | digoal200164.zmf | pg_system | /data01/gpdata/gpseg710 | 8 | p | 40002 | digoal200164.zmf | pg_system | /data01/gpdata/gpseg811 | 9 | p | 40000 | digoal204016.zmf | pg_system | /data01/gpdata/gpseg912 | 10 | p | 40001 | digoal204016.zmf | pg_system | /data01/gpdata/gpseg1013 | 11 | p | 40002 | digoal204016.zmf | pg_system | /data01/gpdata/gpseg1114 | 12 | p | 40000 | digoal204063.zmf | pg_system | /data01/gpdata/gpseg1215 | 13 | p | 40001 | digoal204063.zmf | pg_system | /data01/gpdata/gpseg1316 | 14 | p | 40002 | digoal204063.zmf | pg_system | /data01/gpdata/gpseg1417 | 15 | p | 40003 | digoal193096.zmf | pg_system | /data01/gpdata/gpseg1518 | 16 | p | 40003 | digoal199092.zmf | pg_system | /data01/gpdata/gpseg1619 | 17 | p | 40003 | digoal200164.zmf | pg_system | /data01/gpdata/gpseg1720 | 18 | p | 40003 | digoal204016.zmf | pg_system | /data01/gpdata/gpseg1821 | 19 | p | 40003 | digoal204063.zmf | pg_system | /data01/gpdata/gpseg1922 | 20 | p | 40000 | digoal209198.zmf | pg_system | /data01/gpdata/gpseg2223 | 21 | p | 40001 | digoal209198.zmf | pg_system | /data01/gpdata/gpseg2324 | 22 | p | 40002 | digoal209198.zmf | pg_system | /data01/gpdata/gpseg2425 | 23 | p | 40003 | digoal209198.zmf | pg_system | /data01/gpdata/gpseg25
(25 rows)[参考]
gpfilespace -h
Greenplum 表空间和filespace的用法相关推荐
- Greenplum表空间管理
本文介绍如何创建表空间,如何在表空间上创建数据库,如何查询表空间和database: 目录 1 表空间的概述 2 创建filespace 2.1 检查当前的配置情况 2.2 合理利用高速盘的方案 2. ...
- Greenplum创建表空间
Greenplum中创建表空间的方式和pg中有较大不同,因为gp中有表空间(tablespace)和文件空间(filespace)两个概念. 我们可以这样去理解,因为在gp中master和segmen ...
- oracle建库及plsql建表空间的用法
所有程序->ORACLE-JHEMR----------->配置和移植工具----->DataBase Configuration Assistant-------中间就需要改一个数 ...
- mysql 表空间收缩_mysql表碎片清理和表空间收缩
mysql表碎片清理和表空间收缩(即清理碎片后report_site_day.ibd文件磁盘空间减小,该方案基于独立表空间存储方式) OPTIMIZETABLE [tablename],当然这种方式只 ...
- Oracle传输表空间
一.简介 可传输表空间(还有个集)最大的优势是其速度比export/import或unload/load要快的多.因为可传输表空间主要是复制数据文件到目标路径,然后再使用export/import或D ...
- oracle元数据存储在表空间,[Oracle] dbms_metadata.get_ddl 的使用方法总结
dbms_metadata.get_ddl()用于获取对象的DDL,其具体用法如下. 注意:在sqlplus里,为了更好的展示DDL,需要设置如下参数: set line 200 set pagesi ...
- oracle表空间于表数据啥意思,初识Oracle表空间与数据文件
初识Oracle表空间与数据文件,描述oracle数据库的逻辑结构,创建表空间,改变表空间大小,为临时段分配空间,改变表空间状态,改 学习目标 1.描述Oracle数据库的逻辑结构 2.创建表空间 3 ...
- Oracle之同义词,DBLINK,表空间
昨天讲了物化视图了,今天讲一下同义词的这个概念 就是synonym,反正这个我叫习惯了,这个同义词了解过吗,这个可能了解的不多吧,MYSQL里面有同义词的概念吗,给我点反馈,你用MYSQL里面有同义词 ...
- Oracle 用户、表空间、授权、备份、导入等操作相关
一.基础操作 闲来无事,整理oracle数据库相关操作,以后备用..... ps: satp 为用户 satp_data 为表空间 1 1.删除表空间 2 DROP TABLESPACE satp ...
最新文章
- ASP.NET MVC 异常Exception拦截
- yum的基本使用(命令+案列)
- VC绘制控件如何防止闪烁
- B端产品经理知识框架
- Cocos2dx 3.2移植到Android 完全可行的方法
- FBX模型格式和FbxSDK
- Contrastive learning的学习
- 造成主板通电自动开机的原因 【转】
- 【C/C++调整线程优先级】
- oracle 表添加一列
- [ app.json 文件内容错误] app.json: window.navigationBarTextStyle 字段需为 black,white【已解决】
- react-router v4 路由改变页面不刷新
- 博弈:阿里巴巴与雅虎
- 动手深度学习——Pytorch 入门语法一
- c语言第二单元测试,知到计算机程序设计C语言第二单元章节测试答案
- Debian 官方下载地址
- FofaMap云查询版
- Prometheus Operator 通过additional 添加target
- Delphi打造日常生活支出记账本『罗斌原创』
- 使用PTGui用360度全景图制作Unity3D天空盒
热门文章
- 【Flashback】Flashback Database闪回数据库功能实践
- 《ASP.NET 1.1入门经典—— VISUAL C# .NET 2003编程篇》学习笔记和心得 - 第十章
- matlab均线程序化交易,【策略分享】Matlab量化交易策略源码分享
- MySQL服务的启动与停止-使用命令行工具
- NIO和BIO如何影响应用程序的设计-数据处理
- Java并发编程的基础-为什么要复位
- IDEA 集成Lombok 插件-安装插件
- Spring简介-Spring发展历程
- 从源码深处体验Spring核心技术--面试中IOC那些鲜为人知的细节
- ES6新特性之扩展运算符