一、环境准备

dolphinscheduler1.3.9伪分布式集群
zookeeper:3.7.1容器
mysql:5.7容器

二、构建镜像

(一)、dolphinscheduler镜像

1、编写build.sh(方便后面部署迁移)

#!/bin/sh
#Actively load user env
if [ -f "~/.bashrc" ];thenecho "Warning! user bashrc file does not exist."
elsesource ~/.bashrc
fishellDir=`dirname $0`
workDir=`cd ${shellDir};pwd`APP_HOME=${workDir}/app
APP_NAME=dolphinscheduler
APP_ORI_NAME=apache-dolphinscheduler-1.3.9-bin
APP_SQL_PATH="${APP_HOME}/sql/dolphinscheduler_mysql.sql"
DB_TYPE=mysql
DB_HOST=db-mysql:3306
DB_NAME=dolphinscheduler
USERNAME=test
PASSWORD=test
ZKQUORUM=zookeeper:2181
INSTALLPATH=/opt/soft/dolpherscheduler
DEPLOYUSER=hadoop
ADDRESS=localhost
WORKERS=localhost:default#1 Init Database
cp -rf mysql-connector-java-5.1.47.jar ${APP_HOME}/lib/
MYSQL_DIR=${workDir}
MYSQL_INIT_SQL_PATH=${MYSQL_DIR}/init.sql
echo "create database if not exists ${APP_NAME};" >> ${MYSQL_INIT_SQL_PATH}
echo "GRANT ALL PRIVILEGES ON ${APP_NAME}.* TO '${USERNAME}'@'%' IDENTIFIED BY '${PASSWORD}';" >> ${MYSQL_INIT_SQL_PATH}
echo "GRANT ALL PRIVILEGES ON ${APP_NAME}.* TO '${USERNAME}'@'localhost' IDENTIFIED BY '${PASSWORD}';" >> ${MYSQL_INIT_SQL_PATH}
echo "flush privileges;" >> ${MYSQL_INIT_SQL_PATH}#2 Modify Properties\\
DATASOURCE_PROPERTIES="${APP_HOME}/conf/datasource.properties"
sed -i "s/spring\.datasource\.driver-class-name=org.postgresql.Driver/spring.datasource.driver-class-name=com.mysql.jdbc.Driver/g" ${DATASOURCE_PROPERTIES}
sed -i "s/spring\.datasource\.url=jdbc:postgresql:\/\/127\.0\.0\.1:5432\/dolphinscheduler/spring.datasource.url=jdbc:mysql:\/\/${DB_HOST}\/dolphinscheduler?useUnicode=true\&characterEncoding=UTF-8\&useSSL=false/g" ${DATASOURCE_PROPERTIES}
sed -i "s/spring\.datasource\.username=root/spring.datasource.username=${USERNAME}/g" ${DATASOURCE_PROPERTIES}
sed -i "s/spring\.datasource\.password=root/spring.datasource.password=${PASSWORD}/g" ${DATASOURCE_PROPERTIES}BOOTSTRAP_PROP_FILE="${APP_HOME}/conf/config/install_config.conf"
sed -ri "s![#]?(dbtype=)\S*!\1${DB_TYPE}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(dbhost=)\S*!\1${DB_HOST}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(dbname=)\S*!\1${DB_NAME}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(username=)\S*!\1${USERNAME}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(password=)\S*!\1${PASSWORD}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(zkQuorum=)\S*!\1${ZKQUORUM}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(installPath=)\S*!\1${INSTALLPATH}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(deployUser=)\S*!\1${DEPLOYUSER}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(ips=)\S*!\1${ADDRESS}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(masters=)\S*!\1${ADDRESS}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(workers=)\S*!\1${WORKERS}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(alertServer=)\S*!\1${ADDRESS}!g" ${BOOTSTRAP_PROP_FILE}
sed -ri "s![#]?(apiServers=)\S*!\1${ADDRESS}!g" ${BOOTSTRAP_PROP_FILE}#3 Start Server
#echo "build docker"
#sudo docker build -t heracles/${APP_NAME}:latest .
#rm -rf $APP_HOME

2、编写DockerFile

FROM datawarehouse
#datawarehouse是自己构建的镜像,里面包含jdk、hive、spark等
COPY app /dolphinschedulerENV APP_HOME /dolphinscheduler
ENV DOCKER true
RUN sh /dolphinscheduler/scripts/create_dolphinscheduler.sh
RUN sh /dolphinscheduler/install.sh
EXPOSE 12345
ENTRYPOINT ["sh", "/dolphinscheduler/bin/start-all.sh"]

sh build.sh

(二)、mysql镜像(直接拉取)

docker pull mysql:5.7

同时下载mysql-connector-java-5.1.47.jar
当前目录下新建my.cnf文件,后面挂载到mysql容器中

[client]
default-character-set=utf8
[mysqld]
user=mysql
default_authentication_plugin=mysql_native_password
character_set_server=utf8
collation_server=utf8_general_ci
max-connections=2000
secure_file_priv=''
log_output=FILE
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION

(三)、zookeeper镜像(直接拉取)

docker pull zookeeper:3.7.1

三、容器编排

使用docker-compose.yml文件把三个容器放置于同一个网络中

version: '3'
services:dolphinscheduler:build: .depends_on:- mysql- zookeepernetworks:- heracles-backend- heracles-dbmysql:image: mysql:5.7restart: alwaysprivileged: trueenvironment:MYSQL_ROOT_PASSWORD: 'Fintech!2023'volumes:- ./my.cnf:/etc/my.cnf- ./init.sql:/docker-entrypoint-initdb.d/init.sql#把当前目录下init.sql文件挂载到docker-entrypoint-initdb.d/init.sql下,docker-entrypoint-initdb.d是一个特殊的目录，它位于Docker容器中运行的MySQL、PostgreSQL、MongoDB等数据库镜像的根目录下。这个目录中的任何脚本都将在容器第一次启动时执行，并且只会在该容器第一次启动时执行一次。networks:heracles-db:aliases:- db-mysqlzookeeper:image: zookeeper:3.7.1networks:- heracles-backendnetworks:heracles-db:heracles-backend:

docker-compose up -d

docker学习:docker部署dolphinscheduler1.3.9伪分布式集群相关推荐

linux上部署最新版本zookeeper伪分布式集群
1.环境准备 centos7系统,VM安装centos可参考还不会使用linux?快来通过VMware安装centos系统吧~ zookeeper安装包 SecureCRT 2.zookeeper简介 ...
Hadoop伪分布式集群的安装部署
Hadoop伪分布式集群的安装部署Hadoop伪分布式集群的安装部署首先可以为Linux虚拟机搭建起来的最初状态做一个快照,方便后期搭建分布式集群时多台Linux虚拟机的准备. 一.如何为虚拟机做快 ...
Linux 部署Hadoop伪分布式集群教程
首先:我们需要下载一些关于Hadoop伪分布式集群需要的工具与tar包链接: https://pan.baidu.com/s/1oUw1jDCxfghWsnaWauSHKg 提取码:6s5a 接下来 ...
hadoop搭建伪分布式集群（centos7+hadoop-3.1.1）
原文地址:https://www.cnblogs.com/zhengna/p/9316424.html Hadoop三种安装模式搭建伪分布式集群准备条件第一部分安装前部署 1.查看虚拟机版本 2 ...
伪分布式集群、完全分布式搭建步骤详情
** 伪分布式集群.完全分布式搭建步骤详情 4.1 伪分布式模式介绍 1.特点- 在一台机器上安装,使用的是分布式思想,即分布式文件系统,非本地文件系统.- Hdfs涉及到的相关守护进程(nameno ...
zookeeper伪分布式集群搭建
zookeeper集群搭建注意点: 配置数据文件myid1/2/3对应server.1/2/3 通过zkCli.sh -server [ip]:[port]检测集群是否配置成功. 第一步:首先我们将我 ...
ZooKeeper伪分布式集群安装及使用
为什么80%的码农都做不了架构师?>>> ZooKeeper伪分布式集群安装及使用让Hadoop跑在云端系列文章,介绍了如何整合虚拟化和Hadoop,让Hadoop集群跑在V ...
ZooKeeper伪分布式集群安装
为什么80%的码农都做不了架构师?>>> 获取ZooKeeper安装包下载地址:http://apache.dataguru.cn/zookeeper 选择一个稳定版本进行下 ...
Hadoop单机/伪分布式集群搭建（新手向）
此文已由作者朱笑笑授权网易云社区发布. 欢迎访问网易云社区,了解更多网易技术产品运营经验. 本文主要参照官网的安装步骤实现了Hadoop伪分布式集群的搭建,希望能够为初识Hadoop的小伙伴带来借鉴意 ...

docker学习:docker部署dolphinscheduler1.3.9伪分布式集群

docker&dolphinscheduler

一、环境准备

二、构建镜像

(一)、dolphinscheduler镜像

(二)、mysql镜像(直接拉取)

(三)、zookeeper镜像(直接拉取)

三、容器编排

docker学习:docker部署dolphinscheduler1.3.9伪分布式集群相关推荐

最新文章

热门文章