微服务集成SkyWalking

title: 微服务集成SkyWalking
date: 2019-11-24 09:59:21
categories:

微服务
tags:
SkyWalking

最近做的一个项目使用了Spring Cloud Alibaba，随着服务系统的增多，调用连也越来越复杂，好几天不碰，就有点忘了，所以想集成一个APM系统进来，没想到遇到很多坑，边学习边记录，最开始学习netflix那一套的时候，玩过zipkin，觉得界面有点丑。。。想到SkyWalking是国人写的，文档多点，就尝试了下。

架构

SkyWalking的架构图就不偷了，它主要分成4大模块，这些模块也是我们需要安装的。

Agent（客户端收集器）：使用 JavaAgent 做字节码植入，无侵入式的收集，并通过 HTTP 或者 gRPC 方式发送数据到 SkyWalking Collector。
Oap（服务端分析）：链路数据收集器，对 agent 传过来的数据进行整合分析处理并落入相关的数据存储中。
Storage（存储）： SkyWalking 的存储，Mysql、ES、H2存储介质进行数据存储。
UI：Web 可视化平台，用来展示落地的数据。

安装

使用官方的docker镜像部署，docker-compose文件

# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.version: '3.3'
services:elasticsearch:image: docker.elastic.co/elasticsearch/elasticsearch:6.4.3container_name: elasticsearchrestart: alwaysports:- 9201:9200- 9301:9300environment:discovery.type: single-nodeTZ: Asia/Shanghaiulimits:memlock:soft: -1hard: -1oap:image: apache/skywalking-oap-servercontainer_name: oapdepends_on:- elasticsearchlinks:- elasticsearchrestart: alwaysports:- 11800:11800- 12800:12800environment:SW_STORAGE: elasticsearchSW_STORAGE_ES_CLUSTER_NODES: 10.0.12.4:9201TZ: Asia/Shanghaiui:image: apache/skywalking-uicontainer_name: uidepends_on:- oaplinks:- oaprestart: alwaysports:- 8480:8080environment:SW_OAP_ADRESS: 11.0.12.4:12800TZ: Asia/Shanghai

自己构建镜像

https://github.com/apache/skywalking/blob/master/docs/en/guides/How-to-build.md

下载源码，下载依赖的git子模块

git clone https://github.com/apache/skywalking.git
git submodule init
git submodule update

编译

mvn clean package -Dmaven.test.skip=true

构建docker镜像

# cd到docker目录下
[root@hmaster docker]#
# 分别进入oap、ui目录
[root@hmaster docker]# ls
config  docker-compose.yml  d.yml  oap  README.md  ui
# 将Dockerfile.oap改名为Dockerfile
[root@hmaster oap]# mv Dockerfile.oap Dockerfile
# 粘贴apache-skywalking-apm-bin.tar.gz到同一个上下文中
[root@hmaster oap]# cp ../../dist/apache-skywalking-apm-bin.tar.gz .
# 构建
[root@hmaster oap]# docker build -t skywalking/oap .
# ui 同理

查看下

[root@hmaster oap]# docker images | grep skywalking
skywalking/ui                                   latest              d7f95be7e9f1        23 minutes ago      803MB
skywalking/oap                                  latest              53a8f7b758a2        35 minutes ago      869MB

docker-compose

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.version: '3.3'
services:elasticsearch:image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2container_name: elasticsearchrestart: alwaysports:- 9200:9200- 9300:9300environment:discovery.type: single-nodeTZ: Asia/Shanghaioap:image: skywalking/oapcontainer_name: oapdepends_on:- elasticsearchlinks:- elasticsearchrestart: alwaysports:- 11800:11800- 12800:12800volumes:- ./config:/skywalking/config:roenvironment:TZ: Asia/Shanghaiui:image: skywalking/uicontainer_name: uidepends_on:- oaplinks:- oaprestart: alwaysports:- 8480:8080environment:collectorListOfServers: oap:12800TZ: Asia/Shanghai

官方编译好的版本：

version: '3.3'
services:elasticsearch:image: wutang/elasticsearch-shanghai-zone:6.3.2container_name: elasticsearchrestart: alwaysports:- 9200:9200- 9300:9300environment:cluster.name: elasticsearch

下载，解压

http://skywalking.apache.org/downloads/

[root@hmaster apache-skywalking-apm-bin]# ll
总用量 112
drwxrwxr-x 8 mysql www   4096 9月   9 02:14 agent
drwxr-xr-x 2 root  root  4096 11月 23 10:10 bin
drwxr-xr-x 2 root  root  4096 11月 23 10:45 config
-rwxrwxr-x 1 mysql www  28903 9月   9 02:04 LICENSE
drwxrwxr-x 3 mysql www   4096 11月 23 10:10 licenses
drwxr-xr-x 2 root  root  4096 11月 23 10:18 logs
drwxr-xr-x 2 root  root  4096 11月 23 10:23 mesh-buffer
-rwxrwxr-x 1 mysql www  31850 9月   9 02:04 NOTICE
drwxrwxr-x 2 mysql www  12288 9月   9 02:22 oap-libs
-rw-rw-r-- 1 mysql www   1978 9月   9 02:04 README.txt
drwxr-xr-x 3 root  root  4096 11月 23 10:23 trace-buffer
drwxr-xr-x 2 root  root  4096 11月 24 09:54 webapp

修改配置，进入config文件夹，修改application.yml，注释掉h2，使用es，按需修改。

storage:elasticsearch:nameSpace: ${SW_NAMESPACE:"CollectorDBCluster"}clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:127.0.0.1:9201}
#    protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}
#    trustStorePath: ${SW_SW_STORAGE_ES_SSL_JKS_PATH:"../es_keystore.jks"}
#    trustStorePass: ${SW_SW_STORAGE_ES_SSL_JKS_PASS:""}
#    user: ${SW_ES_USER:""}
#    password: ${SW_ES_PASSWORD:""}indexShardsNumber: ${SW_STORAGE_ES_INDEX_SHARDS_NUMBER:2}indexReplicasNumber: ${SW_STORAGE_ES_INDEX_REPLICAS_NUMBER:0}
#    # Those data TTL settings will override the same settings in core module.
#    recordDataTTL: ${SW_STORAGE_ES_RECORD_DATA_TTL:7} # Unit is day
#    otherMetricsDataTTL: ${SW_STORAGE_ES_OTHER_METRIC_DATA_TTL:45} # Unit is day
#    monthMetricsDataTTL: ${SW_STORAGE_ES_MONTH_METRIC_DATA_TTL:18} # Unit is month
#    # Batch process setting, refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.htmlbulkActions: ${SW_STORAGE_ES_BULK_ACTIONS:1000} # Execute the bulk every 1000 requestsflushInterval: ${SW_STORAGE_ES_FLUSH_INTERVAL:10} # flush the bulk every 10 seconds whatever the number of requestsconcurrentRequests: ${SW_STORAGE_ES_CONCURRENT_REQUESTS:2} # the number of concurrent requests
#    metadataQueryMaxSize: ${SW_STORAGE_ES_QUERY_MAX_SIZE:5000}
#    segmentQueryMaxSize: ${SW_STORAGE_ES_QUERY_SEGMENT_SIZE:200}
#  h2:
#    driver: ${SW_STORAGE_H2_DRIVER:org.h2.jdbcx.JdbcDataSource}
#    url: ${SW_STORAGE_H2_URL:jdbc:h2:mem:skywalking-oap-db}
#    user: ${SW_STORAGE_H2_USER:sa}
#    metadataQueryMaxSize: ${SW_STORAGE_H2_QUERY_MAX_SIZE:5000}
#  mysql:
#    metadataQueryMaxSize: ${SW_STORAGE_H2_QUERY_MAX_SIZE:5000}

Skywalking中默认使用的端口有8080、11800、12800，如需修改，可以修改config目录中的application.yml和webapp目录中的webapp.yml。

进入bin目录中，执行命令./startup.sh。

客户端Agent配置

服务端部署好了，需要在程序里部署客户端Agent。

我们只需要把整个/agent目录挂载到各个微服务中就差不多了。

[root@hmaster agent]# ls
activations  bootstrap-plugins  config  logs  optional-plugins  plugins  skywalking-agent.jar

Dockerfile

FROM java:8
# 将运行脚本加入到容器中
COPY startup.sh /shell/startup.sh
# 赋予运行权限
RUN chmod +x /shell/startup.sh
# 指定容器时区:东八区
ENV TZ=Asia/Shanghai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
# 创建jar包目录
RUN mkdir jar
RUN mkdir agent
# 挂载 jar,agent目录
VOLUME ["/jar","/agent"]
# 进行运行脚本
CMD [ "sh","/shell/startup.sh" ]

startup.sh


i#!/bin/bash
server_name="business-config-service-0.0.1-SNAPSHOT"
kill -s 9 `ps -aux | grep $server_name | awk '{print $2}'`
# 启动，添加agent参数，skywalking-oap地址，服务名
java -javaagent:agent/skywalking-agent.jar -Dskywalking.agent.service_name=config-business -Dskywalking.collector.backend_service=10.0.12.4:11800 -jar jar/$server_name.jar

样例docker-compose，将容器内的agent目录挂载到skywalking目录下的agent目录

version: "3.5"
services:gateway:image: gatewaycontainer_name: gatewayvolumes:- /usr/local/docker/oo1/jars/gateway:/jar- /usr/local/apache-skywalking-apm-bin/agent:/agentports:- "8889:8889"