docker的私有仓库搭建、compose项目、swarm集群、一键部署docker节点

1.私有仓库搭建

首先需要一个registry 镜像

[root@foundation24 docker]# docker images registry
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
registry            latest              b2b03e9146e1        6 weeks ago         33.3 MB
registry            2.3.1               83139345d017        2 years ago         166 MB

挂载registry镜像

[root@foundation24 opt]# docker run -d -p 5000:5000 -v /opt/registry:/var/lib/registry registry:2.3.1
17dd3353b41bf468d4c72e291db6190fd96ee2997e94bf351bef7e8247ae5a05

域名解析:

[root@foundation24 ~]# vim /etc/hosts

查看信息

[root@foundation24 opt]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
17dd3353b41b        registry:2.3.1      "/bin/registry /et..."   35 seconds ago      Up 33 seconds       0.0.0.0:5000->5000/tcp   elated_williams

将本地的nginx镜像改名为westos.org:5000/nginx

[root@foundation24 opt]# docker tag nginx westos.org:5000/nginx   ##改名字

将本地nginx镜像名字改为localhost:5000/nginx

[root@foundation24 opt]# docker tag nginx localhost:5000/nginx

本地镜像上传：

[root@foundation24 opt]# docker push localhost:5000/nginx   ##上送
The push refers to a repository [localhost:5000/nginx]
08d25fa0442e: Pushed
a8c4aeeaa045: Pushed
cdb3f9544e4c: Pushed
latest: digest: sha256:2de9d5fc6585b3f330ff5f2c323d2a4006a49a476729bbc0910b695771526e3f size: 948

westos.org:5000/nginx上传

[root@foundation24 opt]# docker push westos.org:5000/nginx
The push refers to a repository [westos.org:5000/nginx]
Get https://westos.org:5000/v1/_ping: http: server gave HTTP response to HTTPS client

以上实验说明，这样配置的私有仓库只能仅限与本地操作，无法分享

删除：

[root@foundation24 opt]# docker rmi localhost:5000/nginx
root@foundation24 opt]# docker rmi westos.org:5000/nginx

配置可以在外网访问的私有仓库

[root@foundation24 docker]# pwd
/tmp/docker

配置秘钥

[root@foundation24 docker]# openssl req -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key -x509 -days 365 -out certs/domain.crt
Generating a 4096 bit RSA private key

生成秘钥和开放443端口

[root@foundation24 docker]# docker run -d \
>   --restart=always \
>   --name registry \
>   -v `pwd`/certs:/certs \
>   -e REGISTRY_HTTP_ADDR=0.0.0.0:443 \
>   -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
>   -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
>   -p 443:443 \
>   registry:2
ac4b23c63602603c98b2e049b56e1565c2f15c69371ae235e85db76efaa39e1

查看信息

[root@foundation24 docker]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                            NAMES
1957797318eb        registry:2          "/entrypoint.sh /e..."   About a minute ago   Up About a minute   0.0.0.0:443->443/tcp, 5000/tcp   registry

查看端口：

[root@foundation24 registry]# iptables -t nat -nL

[root@foundation24 registry]# netstat -antlp |grep :443
tcp6       0      0 :::443                  :::*                    LISTEN      26762/docker-proxy

创建秘钥目录：

[root@foundation24 registry]# cd /etc/docker/
[root@foundation24 docker]# mkdir certs.d
[root@foundation24 docker]# cd certs.d/
[root@foundation24 certs.d]# mkdir westos.org
[root@foundation24 certs.d]# ls
westos.org
[root@foundation24 certs.d]# cd westos.org/
[root@foundation24 westos.org]# ls

将生成的秘钥复制过来

[root@foundation24 westos.org]# cp /tmp/docker/certs/domain.crt ./ca.crt
[root@foundation24 westos.org]# ls
ca.crt
[root@foundation24 westos.org]# ll
total 4
-rw-r--r-- 1 root root 2098 Aug 21 18:09 ca.crt

测试：

[root@foundation24 docker]# docker push westos.org/nginx
The push refers to a repository [westos.org/nginx]
08d25fa0442e: Pushed
a8c4aeeaa045: Pushed
cdb3f9544e4c: Pushed
latest: digest: sha256:2de9d5fc6585b3f330ff5f2c323d2a4006a49a476729bbc0910b695771526e3f size: 948

给仓库设置密码和用户：

设置两个用户和密码

[root@foundation24 docker]# mkdir auth
[root@foundation24 docker]# docker run --entrypoint htpasswd registry:2 -Bbn whx westos > auth/htpasswd
[root@foundation24 docker]# docker run --entrypoint htpasswd registry:2 -Bbn admin admin >> auth/htpasswd   ##追加

会在当前目录下的auth目录中生成htpasswd

[root@foundation24 docker]# cat auth/htpasswd
whx:$2y$05$IZRARKJ/xcRSztM6aOyLVOcL.WlLADUkva.mT3xZhr6JS/Mqi7lvyadmin:$2y$05$rHSwKTKKgGnFrF.zUzmEMOCPBff800Ksyp0Ji8KqLIC19wm.eWtiW

开启443端口并添加密码信息

[root@foundation24 docker]# docker run -d    --restart=always    --name registry    -v `pwd`/certs:/certs    -e REGISTRY_HTTP_ADDR=0.0.0.0:443    -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt    -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key  -v `pwd`/auth:/auth -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm"  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd  -p 443:443    registry:2
065001d48018c262b4a5ce8029f5f56374bc951622a6be0965412d1b2439c02e

查看容器信息


[root@foundation24 docker]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                            NAMES
065001d48018        registry:2          "/entrypoint.sh /e..."   5 seconds ago       Up 3 seconds        0.0.0.0:443->443/tcp, 5000/tcp   registry

用admin用户登陆

[root@foundation24 docker]# docker login -u admin -p admin westos.org
Login Succeeded

查看端口

[root@foundation24 docker]# netstat -antlp |grep :443
tcp6       0      0 :::443                  :::*                    LISTEN      8053/docker-proxy

只有记录了admin用户的信息，才能上传或下载

[root@foundation24 ~]# cat .docker/config.json
{"auths": {"westos.org": {"auth": "YWRtaW46YWRtaW4="}}

二.comepose项目

原理图：

Docker Compose 将所管理的容器分为三层,工程(project),服务(service)以及容器(contaienr)。Docker Compose 运行的目录下的所有文件(dockercompose.yml, extends 文
件或环境变量文件等)组成一个工程,若无特殊指定工程名即为当前目录名。一个工程当中可包含多个服务,每个服务中定义了容器运行的镜像,参数,依赖。一个服务当中可包括多个容器实例,Docker Compose 并没有解决负载均衡的问题,因此需要借助其他工具实现服务发现及负载均衡。
Docker Compose 是一个用来创建和运行多容器应用的工具。使用 Compose首先需要编写Compose 文件来描述多个容器服务以及之间的关联,然后通过令根据配置启动所有的容器。
Dockerfile 可以定义一个容器,而一个 Compose 的模板文件(YAML 格式)可以定义一个包含多个相互关联容器的应用。Compose 项目使用 python 编写,于后面的实验中我们将学习的 Docker API 实现。

此时可以使用 docker build 创建 web 镜像,然后在 Compose 的配置文件中指定镜像名称
为 web,也可以在配置文件中直接指定 Dockerfile, Compose 会自动 build 镜像

使用comepose实现负载均衡


[root@foundation24 docker]# mkdir compose
[root@foundation24 docker]# cd compose/
[root@foundation24 compose]# pwd
/tmp/docker/compose
[root@foundation24 compose]# ls
[root@foundation24 compose]# vim docker-compose.yml
root@foundation24 compose]# cat docker-compose.yml
apache:image: rhel7:v1expose:- 80                                 ##在容器中http的端口
    volumes:- ./web:/var/www/html            ##当前目录下的web目录对应容器中http的默认目录
nginx:image: nginxexpose:- 80
haproxy:image: haproxyvolumes:- ./haproxy:/usr/local/etc/haproxy        ##将配置文件导入容器
    links:- apache                       ##调用apache
        - nginx                         ##调用nginx
    ports:- "8080:80"                   ##端口映射物理机：8080
    expose:- 80

编写haproxy配置文件

[root@foundation24 compose]# cp -r ../web/ .
[root@foundation24 compose]# ls
docker-compose.yml  web
[root@foundation24 compose]# mkdir haproxy
[root@foundation24 compose]# cd haproxy/
[root@foundation24 haproxy]# vim haproxy.cfg
globallog 127.0.0.1 local0log 127.0.0.1 local1 notice
defaultslog globalmode httpoption httplogoption dontlognulltimeout connect 5000mstimeout client 50000mstimeout server 50000msstats uri /status
frontend balancerbind 0.0.0.0:80default_backend web_backends
backend web_backendsbalance roundrobinserver weba apache:80 check            ##是因为容器中ip不明确直接用名称代替server webb nginx:80 check

添加compose服务：

[root@foundation24 docker]# cd /usr/local/bin/
[root@foundation24 bin]# ls
charm  rht-vmctl  rht-vmicons  rht-vmsetkeyboard
[root@foundation24 bin]# lftp 172.25.254.251
lftp 172.25.254.251:~> cd pub/docs/docker/
lftp 172.25.254.251:/pub/docs/docker> get docker-compose-Linux-x86_64-1.22.0
11750136 bytes transferred in 2 seconds (4.75M/s)
lftp 172.25.254.251:/pub/docs/docker> quit
[root@foundation24 bin]# chmod +x docker-compose-Linux-x86_64-1.22.0
[root@foundation24 bin]# ln -s docker-compose-Linux-x86_64-1.22.0  docker-compose
[root@foundation24 bin]# ll docker-compose
lrwxrwxrwx 1 root root 34 Aug 22 11:44 docker-compose -> docker-compose-Linux-x86_64-1.22.0[root@foundation24 ~]# docker-compose -v     ##查看版本
docker-compose version 1.22.0, build f46880fe

测试：

[root@foundation24 ~]# cd /tmp/docker/
[root@foundation24 docker]# cd compose/     ##必须在这个目录中
[root@foundation24 compose]# ls
docker-compose.yml  haproxy  web

[root@foundation24 compose]# docker-compose up
Creating compose_nginx_1  ... done
Creating compose_apache_1 ... done
Creating compose_haproxy_1 ... done
Attaching to compose_nginx_1, compose_apache_1, compose_haproxy_1
apache_1   | AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.3. Set the 'ServerName' directive globally to suppress this message

查看监控：

这种方法在你刷新浏览器进行轮询的时候，会有记录,刷新一次，记录一次

Dockerswarm

Swarm 是 Docker 公司在 2014 年 12 月初发布的一套较为简单的工具,用来管理 Docker 集群,它将一群 Docker 宿主机变成一个单一的,虚拟的主机。Swarm 使用标准的 Docker API接口作为其前端访问入口,换言之,各种形式的 Docker Client(docker client in go, docker_py,docker 等)均可以直接与 Swarm 通信。Swarm 几乎全部用 Go 语言来完成开发。
Swarm deamon 只是一个调度器(Scheduler)加路由器(router),Swarm 自己不运行容器,它只是接受 docker 客户端发送过来的请求,调度适合的节点来运行容器,这意味着,即使Swarm 由于某些原因挂掉了,集群中的节点也会照常运行,当 Swarm 重新恢复运行之后,它会收集重建集群信息。下面是 Swarm 的结构图:

环境：
swarm manager ：server2
swarm node ：server2 、server3 、server4
初始化节点：

[root@server2 ~]# docker  swarm init   ##初始化
Swarm initialized: current node (2pvpzju2kud9yud6kq3g78hcb) is now a manager.To add a worker to this swarm, run the following command:docker swarm join \--token SWMTKN-1-2c6jzhvw3qnhhae2cktgda6phs0yyidevr2qves8jre6m84dhy-5tbxzohwg48o3bv5m9pvc5c98 \172.25.24.2:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

连接：

[root@server3 ~]# systemctl start docker
[root@server3 ~]# docker swarm join \
>     --token SWMTKN-1-2c6jzhvw3qnhhae2cktgda6phs0yyidevr2qves8jre6m84dhy-5tbxzohwg48o3bv5m9pvc5c98 \
>     172.25.24.2:2377
This node joined a swarm as a worker

[root@server4 ~]# systemctl  start docker
[root@server4 ~]# docker swarm join \
>     --token SWMTKN-1-2c6jzhvw3qnhhae2cktgda6phs0yyidevr2qves8jre6m84dhy-5tbxzohwg48o3bv5m9pvc5c98 \
>     172.25.24.2:2377
This node joined a swarm as a worker.

查看连接节点

[root@server2 ~]# docker node ls
ID                           HOSTNAME  STATUS  AVAILABILITY  MANAGER STATUS
0cfmikoxmslct982aajun1d9f    server4   Ready   Active
2pvpzju2kud9yud6kq3g78hcb *  server2   Ready   Active        Leader
zrjdoa3rsashl1zwjy09ymbz8    server3   Ready   Active

swarm 部署完成，现在开始是service，先在物理机上搭建一个私有仓库，方便虚拟机下载镜像

[root@foundation24 docker]# docker run -d --restart=always --name registry -v `pwd`/certs:/certs -e REGISTRY_HTTP_ADDR=0.0.0.0:443 -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key -p 443:443 registry:2[root@foundation24 westos.org]# docker tag nginx westos.org/nginx[root@foundation24 westos.org]# docker push westos.org/nginx  ##上传

在把物理机的秘钥传给虚拟机：

[root@server2 westos.org]# pwd
/etc/docker/certs.d/westos.org
[root@server2 westos.org]# ls
ca.crt

OK,现在下面给三个节点部署nginx

[root@server2 westos.org]# docker pull westos.org/nginx[root@server2 ~]# docker service create --name nginx --publish 80:80 --replicas 3 westos.org/nginx  ##部署三个

[root@server2 ~]# docker service ls
ID            NAME   MODE        REPLICAS  IMAGE
rjmc5m8mj42v  nginx  replicated  3/3       westos.org/nginx:latest[root@server2 ~]# docker ps
CONTAINER ID        IMAGE                                                                                      COMMAND                  CREATED             STATUS              PORTS               NAMES
9036b934d7cc        westos.org/nginx@sha256:2de9d5fc6585b3f330ff5f2c323d2a4006a49a476729bbc0910b695771526e3f   "nginx -g 'daemon ..."   37 seconds ago      Up 31 seconds       80/tcp

在上传一个监控镜像

[root@foundation24 docker]# docker load < visualizer.tar[root@foundation24 docker]# docker tag  dockersamples/visualizer westos.org/visualizer
[root@foundation24 docker]# docker push westos.org/visualizer

添加监控镜像

[root@server2 ~]# docker pull westos.org/visualizer
[root@server2 ~]# docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
westos.org/nginx        latest              c82521676580        4 weeks ago         109 MB
westos.org/visualizer   latest              17e55a9b2354        11 months ago       148 MB

建立一个监控，监控映射内网80端口的8080端口

root@server2 ~]# docker service create --name=viz --publish=8080:8080/tcp --constraint=node.role==manager --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock westos.org/visualizer                ##添加监控，监控8080端口
i23942znpg4pgql2sbwedrtn0

[root@server2 ~]# docker service ls
ID            NAME   MODE        REPLICAS  IMAGE
i23942znpg4p  viz    replicated  1/1       westos.org/visualizer:latest
rjmc5m8mj42v  nginx  replicated  3/3       westos.org/nginx:lates

浏览器查看：

swarm会有自动恢复重新配置nginx的功能
停下server4的nginx，然后会自动恢复

root@server4 westos.org]# docker ps
CONTAINER ID        IMAGE                                                                                      COMMAND                  CREATED             STATUS              PORTS               NAMES
dda8e4642212        westos.org/nginx@sha256:2de9d5fc6585b3f330ff5f2c323d2a4006a49a476729bbc0910b695771526e3f   "nginx -g 'daemon ..."   26 minutes ago      Up 26 minutes       80/tcp              nginx.1.etxy7gnjx93oybeynt9qht5jx
[root@server4 westos.org]# docker stop nginx.1.etxy7gnjx93oybeynt9qht5jx
nginx.1.etxy7gnjx93oybeynt9qht5jx
[root@server4 westos.org]# docker ps
CONTAINER ID        IMAGE                                                                                      COMMAND                  CREATED             STATUS              PORTS               NAMES
59e008970b8e        westos.org/nginx@sha256:2de9d5fc6585b3f330ff5f2c323d2a4006a49a476729bbc0910b695771526e3f   "nginx -g 'daemon ..."   54 seconds ago      Up 48 seconds       80/tcp              nginx.1.rqldxkv8zij9fbeu6vfz69nfk

swarm的滚动更新

将线程增加到30个

[root@server2 ~]# docker service  scale nginx=30   ##先增加到30个nginx

透过监控可以查看到每个机器分配10个nginx

用rhel：v1镜像取更新nginx镜像

[root@server2 ~]# docker service update --image westos.org/rhel7:v1 --update-parallelism 3 --update-delay 10s nginx   ##用  westos.org/rhel7:v1来更新nginx   3个3个的更新，时间间隔为10s

更新完成如下，可以看到每个虚拟机的所有nginx服务都变成了http

透过监视也可以看见里面的镜像全部变成rhel7：v1 ，即http

或者浏览器查看：

docker扩展之一键部署docker节点

需要docker-machine服务

[root@foundation24 docker]# mv docker-machine-Linux-x86_64-1.15.0 /usr/local/bin/
[root@foundation24 docker]# cd /usr/local/bin/
[root@foundation24 bin]# ls
charm                               rht-vmctl
docker-compose                      rht-vmicons
docker-compose-Linux-x86_64-1.22.0  rht-vmsetkeyboard
docker-machine-Linux-x86_64-1.15.0
[root@foundation24 bin]# chmod +x docker-machine-Linux-x86_64-1.15.0
[root@foundation24 bin]# ln -s docker-machine-Linux-x86_64-1.15.0 docker-machine
[root@foundation24 bin]# ll docker-machine
lrwxrwxrwx 1 root root 34 Aug 23 09:05 docker-machine -> docker-machine-Linux-x86_64-1.15.0
[root@foundation24 bin]# docker-machine -v
docker-machine version 0.15.0, build b48dc28d

做免密处理：

[root@foundation24 ~]# ssh-keygen[root@foundation24 ~]# ssh-copy-id 172.25.24.2[root@foundation24 ~]# ssh-copy-id 172.25.24.3[root@foundation24 ~]# ssh-copy-id 172.25.24.4

建立连接：

[root@foundation24 ~]# docker-machine create --driver generic --generic-ip-address=172.25.24.2 server2   [root@foundation24 ~]# docker-machine create --driver generic --generic-ip-address=172.25.24.3 server3[root@foundation24 ~]# docker-machine create --driver generic --generic-ip-address=172.25.24.4 server4

查看连接信息

[root@foundation24 ~]# docker-machine ls
NAME      ACTIVE   DRIVER    STATE     URL                      SWARM   DOCKER        ERRORS
server2   -        generic   Running   tcp://172.25.24.2:2376           v17.03.1-ce
server3   -        generic   Running   tcp://172.25.24.3:2376           v17.03.1-ce
server4   -        generic   Running   tcp://172.25.24.4:2376           v17.03.1-ce

测试：

[root@foundation24 ~]# docker-machine ssh server2 docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[root@foundation24 ~]# docker-machine ssh server3 docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

删除：

root@foundation24 ~]# docker-machine rm server2
About to remove server2
WARNING: This action will delete both local reference and remote instance.
Are you sure? (y/n): y
Successfully removed server2
[root@foundation24 ~]# docker-machine rm server3
About to remove server3
WARNING: This action will delete both local reference and remote instance.
Are you sure? (y/n): y
Successfully removed server3
[root@foundation24 ~]# docker-machine rm server4
About to remove server4
WARNING: This action will delete both local reference and remote instance.
Are you sure? (y/n): y
Successfully removed server4

[root@foundation24 ~]# docker-machine ls
NAME   ACTIVE   DRIVER   STATE   URL   SWARM   DOCKER   ERRORS