【巨杉数据库SequoiaDB】巨杉 Tech | 几分钟实现巨杉数据库容器化部署
2024-04-11 11:28:19
随着业务负载的不断加重，容器化、虚拟化也成为各类在线应用必须要具备的能力。对于分布式数据库，容器化也是提升快速部署、提高运维效率的一个很好的路径。
我们重新优化了 Docker部署的方式，帮助大家更快的上手SequoiaDB集群，本文就将介绍基于 Docker 的SequoiaDB分布式集群快速部署。
1.集群配置
我们将在六个容器中部署一个多节点，高度可用的 SequoiaDB 集群，如下所示:
（本文以 SequoiaDB v3.2.3 版本为例）
该集群包括一个协调器节点、一个目录节点、三个数据组，每个数据组有三个副本数据节点和一个 MySQL 实例节点。
2. 环境
OS : Ubuntu 18Docker Version : 18.09.7Docker Compose Version: 1.25.3Database Version : SequoiaDB 3.2.3MySql Client: Sequoiasql-mysqlCluster Deployment : 1 coordinator, 1 catalog, 3 data nodes, and 1 MySQL instance
2.1 Docker安装
2.1.1 Docker
对于在不同平台上的 Docker 安装，用户可以参阅 Docker 安装指南。
sudo apt-get install -y docker.io2.1.2 Docker-compose
Docker compose是一个用于定义和部署在YAML文件中定义的多容器泊坞应用程序的工具(i.e: docker-compose.yaml)。
sudo curl -L "https://github.com/docker/compose/releases/download/1.25.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-composesudo chmod +x /usr/local/bin/docker-composesudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
2.1.3 MySQL 客户端
安装 MySQL 客户端（如果尚未安装)
sudo apt install mysql-client-core-5.73. SequoiaDB分布式集群部署
定义 SequoiaDB 集群配置 "sequoiadb_containers/docker-compose.yaml"，用户有两个选项可以下载所需的软件包。
选项1：克隆repo地址
（点击文末【阅读原文】前往获取软件包）
选项2：手动下载（保留相同的目录结构)
（点击文末【阅读原文】前往获取软件包）
3.1 部署 SequoiaDB 集群
docker-compose up 命令将:
- 如果没有在本地找到，可前往dockerhub下载镜像
- 创建所需的容器
- 启动容器
- 初始化 SequoiaDB 集群和 MySQL 实例
1）部署集群
cd sequoiadb_containers （或者到docker-compose的目录。yaml文件下载)sudo docker-compose up -d
2）检查集群的状态
$ sudo docker-compose psName Command State Ports--------------------------------------------------------------------------------------------sequoiadb_containers_catalog_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_coord_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data1_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data2_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data3_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_mysql_1 sh -c chmod +x /startup.sh ... Up 0.0.0.0:3310->3310/tcp
3）等待集群被部署和 MySQL 连接到协调器节点，用户可以按如下方式检查集群状态。
# 检查 coordinator 日志$ sudo docker-compose logs coord......coord_1 | ************ Deploy SequoiaDB ************************coord_1 | Create catalog: catalog:11800coord_1 | Create coord: coord:11810coord_1 | Create data: data1:11820coord_1 | Create data: data2:11820coord_1 | Create data: data3:11820coord_1 | Create data: data1:11830coord_1 | Create data: data2:11830coord_1 | Create data: data3:11830coord_1 | Create data: data1:11840coord_1 | Create data: data2:11840coord_1 | Create data: data3:11840coord_1 | SDB Cluster successfully created.# 检查mysql日志$ sudo docker-compose logs mysql......mysql_1 | >>> Waiting for COORD node to come up at (coord:11810)...mysql_1 | >>> Waiting for COORD node to come up at (coord:11810)...mysql_1 | >>> Waiting for COORD node to come up at (coord:11810)...mysql_1 | Successfully reached COORD node at (coord:11810)mysql_1 | COMMAND: /init.sh --port=3310 --coord=coord:11810mysql_1 | Creating SequoiaSQL instance: MySQLInstancemysql_1 | Modify configuration file and restart the instance: MySQLInstancemysql_1 | Restarting instance: MySQLInstancemysql_1 | Opening remote access to user rootmysql_1 | Restarting instance: MySQLInstancemysql_1 | Instance MySQLInstance is created on port 3310, default user is rootmysql_1 | Init command returned: 0
3.2 验证MySQL连接
$ mysql -h 127.0.0.1 -P 3310 -u rootWelcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 4Server version: 5.7.25 Source distributionCopyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql>
验证SequoiaDB配置:
mysql> show variables like 'sequoiadb%';+---------------------------------------+-------------+| Variable_name | Value |+---------------------------------------+-------------+| sequoiadb_bulk_insert_size | 2000 || sequoiadb_conn_addr | coord:11810 || sequoiadb_debug_log | OFF || sequoiadb_execute_only_in_mysql | OFF || sequoiadb_optimizer_select_count | ON || sequoiadb_password | || sequoiadb_replica_size | 1 || sequoiadb_selector_pushdown_threshold | 30 || sequoiadb_use_autocommit | ON || sequoiadb_use_bulk_insert | ON || sequoiadb_use_partition | ON || sequoiadb_user | |+---------------------------------------+-------------+12 rows in set (0.01 sec)mysql> show storage engines;+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+| Engine | Support | Comment | Transactions | XA | Savepoints |+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+| SequoiaDB | DEFAULT | SequoiaDB storage engine(Community). Plugin: eebcb4f, Driver: 3.2.3, BuildTime: | YES | NO | NO || MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO || PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO || BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO || MyISAM | YES | MyISAM storage engine | NO | NO | NO || CSV | YES | CSV storage engine | NO | NO | NO || ARCHIVE | YES | Archive storage engine | NO | NO | NO || InnoDB | YES | Supports transactions, row-level locking, and foreign keys | YES | YES | YES || FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL || MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+10 rows in set (0.00 sec)
3.3 测试
用户可以使用 MySQL 命令创建数据库和表：
mysql> create database sample;Query OK, 1 row affected (0.01 sec)mysql> use sample;Database changedmysql> create table t1 (c1 int);Query OK, 0 rows affected (0.47 sec)mysql> show table status;+------+-----------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-------------+----------+----------------+---------+| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |+------+-----------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-------------+----------+----------------+---------+| t1 | SequoiaDB | 10 | Fixed | 0 | 0 | 0 | 8796093022208 | 131072 | 0 | NULL | NULL | NULL | NULL | utf8mb4_bin | NULL | | |+------+-----------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-------------+----------+----------------+---------+1 row in set (0.03 sec)
4. 集群管理
4.1 stop/start cluster
使用up命令创建 SequoiaDB 集群后，它可以停止并按如下方式启动。
Note:
"stop"命令不会删除容器，因此不会丢失任何数据。
4.1.1 stop 命令
$ sudo docker-compose stopStopping sequoiadb_containers_mysql_1 ... doneStopping sequoiadb_containers_coord_1 ... doneStopping sequoiadb_containers_data2_1 ... doneStopping sequoiadb_containers_data1_1 ... doneStopping sequoiadb_containers_catalog_1 ... doneStopping sequoiadb_containers_data3_1 ... done$ sudo docker-compose psName Command State Ports------------------------------------------------------------------------------sequoiadb_containers_catalog_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_coord_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_data1_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_data2_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_data3_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_mysql_1 sh -c chmod +x /startup.sh ... Exit 137
4.1.2 start 命令
$ sudo docker-compose startStarting catalog ... doneStarting data1 ... doneStarting data2 ... doneStarting data3 ... doneStarting coord ... doneStarting mysql ... done$ sudo docker-compose psName Command State Ports--------------------------------------------------------------------------------------------sequoiadb_containers_catalog_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_coord_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data1_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data2_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data3_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_mysql_1 sh -c chmod +x /startup.sh ... Up 0.0.0.0:3310->3310/tcp# Verity mysql client is up$ sudo docker-compose logs mysqlAttaching to sequoiadb_containers_mysql_1mysql_1 | Starting service sequoiasql-mysql ...mysql_1 | ok. (PID: 38)...mysql_1 | MySql instance is started
4.2 down/up Cluster
4.2.1 down命令
down 命令将停止并删除SequoiaDB集群部署的容器。
Note：
down命令应该谨慎执行，因为当容器被破坏时无法回滚/恢复。
$ sudo docker-compose downStopping sequoiadb_containers_mysql_1 ... doneStopping sequoiadb_containers_coord_1 ... doneStopping sequoiadb_containers_data2_1 ... doneStopping sequoiadb_containers_data1_1 ... doneStopping sequoiadb_containers_catalog_1 ... doneStopping sequoiadb_containers_data3_1 ... doneRemoving sequoiadb_containers_mysql_1 ... doneRemoving sequoiadb_containers_coord_1 ... doneRemoving sequoiadb_containers_data2_1 ... doneRemoving sequoiadb_containers_data1_1 ... doneRemoving sequoiadb_containers_catalog_1 ... doneRemoving sequoiadb_containers_data3_1 ... doneRemoving network sequoiadb_containers_sequoiadb_net$ sudo docker-compose psName Command State Ports------------------------------
4.2.2 up命令
up命令将创建并启动为sequoiadb cluser定义的服务（容器，网络）。
$ sudo docker-compose up -dCreating network "sequoiadb_containers_sequoiadb_net" with the default driverCreating sequoiadb_containers_data2_1 ... doneCreating sequoiadb_containers_data3_1 ... doneCreating sequoiadb_containers_catalog_1 ... doneCreating sequoiadb_containers_data1_1 ... doneCreating sequoiadb_containers_coord_1 ... doneCreating sequoiadb_containers_mysql_1 ... done$ sudo docker-compose psName Command State Ports--------------------------------------------------------------------------------------------sequoiadb_containers_catalog_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_coord_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data1_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data2_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data3_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_mysql_1 sh -c chmod +x /startup.sh ... Up 0.0.0.0:3310->3310/tcp# 验证MySQL连接$ mysql -h 127.0.0.1 -P 3310 -u rootWelcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 2Server version: 5.7.25 Source distributionCopyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> show storage engines;+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+| Engine | Support | Comment | Transactions | XA | Savepoints |+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+| SequoiaDB | DEFAULT | SequoiaDB storage engine(Community). Plugin: eebcb4f, Driver: 3.2.3, BuildTime: | YES | NO | NO || MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO || PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO || BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO || MyISAM | YES | MyISAM storage engine | NO | NO | NO || CSV | YES | CSV storage engine | NO | NO | NO || ARCHIVE | YES | Archive storage engine | NO | NO | NO || InnoDB | YES | Supports transactions, row-level locking, and foreign keys | YES | YES | YES || FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL || MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+
不知道您是否已经动手操作了呢？
SequoiaDB后续也将持续推出系列技术教程，帮助大家更快上手分布式数据库，同时更好的理解分布式数据库的原理，大家敬请期待！
【巨杉数据库SequoiaDB】巨杉 Tech | 几分钟实现巨杉数据库容器化部署相关推荐
- 检查集群状态命令_巨杉数据库SequoiaDB巨杉Tech | 四步走，快速诊断数据库集群状态...
1.背景 SequoiaDB 巨杉数据库是一款金融级分布式数据库,包括了分布式 NewSQL.分布式文件系统与对象存储.与高性能 NoSQL 三种存储模式,分别对应分布式在线交易.非结构化数据和内容管 ...
- 回滚机制_【巨杉数据库SequoiaDB】巨杉 Tech | 并发性与锁机制解析与实践
01 概述 数据库是一个多用户使用的共享资源.当多个用户并发地存取数据时,在数据库中就会产生多个事务同时存取同一数据的情况.若对并发操作不加控制就可能会读取和存储不正确的数据,破坏数据库的一致性.加锁 ...
- 【巨杉数据库SequoiaDB】巨杉Tech | 巨杉数据库数据高性能数据导入迁移实践
SequoiaDB 一款自研金融级分布式数据库产品,支持标准SQL和分布式事务功能.支持复杂索引查询,兼容 MySQL.PGSQL.SparkSQL等SQL访问方式.SequoiaDB 在分布式存储功 ...
- 【巨杉数据库SequoiaDB】巨杉 Tech | SequoiaDB SQL实例高可用负载均衡实践
1 前言 在应用程序中,应用配置连接的数据库IP地址和端口号都是固定一个的,当所属IP地址的服务器宕机后,需要人为手工更改IP地址切换数据库服务器.同时当应用接收到成千上万的并发 http 请求时,会 ...
- 【巨杉数据库SequoiaDB】巨杉 Tech | 并发性与锁机制解析与实践
01 概述 数据库是一个多用户使用的共享资源.当多个用户并发地存取数据时,在数据库中就会产生多个事务同时存取同一数据的情况.若对并发操作不加控制就可能会读取和存储不正确的数据,破坏数据库的一致性.加锁 ...
- 如何认识巨杉数据库SequoiaDB整体架构？
SequoiaDB 巨杉数据库集群分为数据库存储引擎与数据库实例. 在当前版本中,SequoiaDB 巨杉数据库支持多达七种不同数据服务实例,包括针对结构化数据的 MySQL.MariaDB.Post ...
- 【巨杉数据库SequoiaDB】巨杉数据库荣获《金融电子化》“金融科技创新奖”
巨杉助力金融科技创新 2019年12月19日,由<金融电子化>杂志社主办.北京金融科技产业联盟协办的"2019中国金融科技年会暨第十届金融科技及服务优秀创新奖颁奖典礼" ...
- 【巨杉数据库SequoiaDB】点燃深秋，巨杉数据库亮相DTC数据技术嘉年华大会
2019年11月15日,第九届数据技术嘉年华大会在北京隆重召开,本次大会以 "开源 • 智能 • 云数据 - 自主驱动发展 创新引领未来" 为主题,探索数据价值,共论智能未来.S ...
- 【巨杉数据库SequoiaDB】巨杉⼯具系列之一 | ⼤对象存储⼯具sdblobtool
近期,巨杉数据库正式推出了完整的SequoiaDB 工具包,作为辅助工具,更好地帮助大家使用和运维管理分布式数据库.为此,巨杉技术社区还将持续推出工具系列文章,帮助大家了解巨杉数据库丰富的工具矩阵. ...
最新文章
- AI一分钟|百度与中国联通成立“5G+AI联合实验室”；SpaceX将向国际空间站发送AI机器人...
- Leetcode-520. 检测大写字母
- EL表达式隐含对象和jstl命名冲突，jstl无法取的值
- java 代码通用结构_java spring代码通用结构-java
- php怎么控制递归多少次,关于一个PHP递归处理统计的问题
- 软件测试部分习题答案
- HDU3247 Resource Archiver（AC自动机+BFS+DP）
- 吴恩达机器学习课程笔记章节二单变量线性回归
- 懒加载Lazy Loading
- linux domino启动和停止命令,在unix平台中通过脚本来启动和停止domino和domino控制器所用的参数_lotus notes...
- 【论文笔记】Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- 初接触设计模式 导航(一)
- PSV 3.60 固化升级到 3.68 破解完全攻略
- 类似QQ的可隐藏的便签工具SNOTE
- 基于Mendix的云上全流程透明性备品备件协同管理
- WPF：MouseDown、MouseUP事件，鼠标按下不起作用
- Linux 操作系统
- 淘宝网首页登录失败原因分析及解决…
- python使用ffmpeg提取mkv视频的字幕srt
- What?小程序居然可以脱离微信环境运行