elasticsearch和mysql的对比

https://blog.csdn.net/qq_21383435/article/details/79323383

Mapping ~ Schema

master: 负责在集群范围内创建/删除索引,将分片分配给这些节点.
data:   用来保存数据和倒排索引,node.data=ture
client: 将节点配置为客户端节点，并充当负载平衡器，将传入的请求路由到集群中的不同节点。node.master和node.data设置为false

https://blog.csdn.net/sdksdk0/article/details/78469190

Index templates

索引可使用预定义的模板进行创建,这个模板称作Index templates。模板设置包括settings和mappings，通过模式匹配的方式使得多个索引重用一个模板，例如：

curl -XPUT localhost:9200/_template/template_1 -d '
{"template" : "te*","settings" : {"number_of_shards" : 1},"mappings" : {"type1" : {"_source" : {"enabled" : false }}}
}

elk设计架构

参考
k8sgithub上 https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch

Elasticsearch最佳实践建议将这些节点分成三个角色：Master 节点 - 仅用于集群管理，没有数据，没有HTTP API
Client 节点 - 用于客户端使用，无数据，  使用HTTP API
Data   节点 - 旨在存储和索引数据，      没有HTTP API

分为 管理区 存储区 读写区

定制镜像

- es
0. 我把镜像推到了dockerhub: lanny/quay.io_pires_docker-elasticsearch-kubernetes:5.6.0
1. 镜像里用到的yaml: https://github.com/lannyMa/docker-elasticsearch-kubernetes
2. 可以安装自定义插件后重新build- kibana
0. 用到这个人的这个镜像 cfontes/kibana-xpack-less:5.5.0 https://hub.docker.com/r/cfontes/kibana-xpack-less/这个人的git(包含了elk的k8s yaml): https://github.com/cfontes/kubernetes-elasticsearch-cluster- k8s的elk集群的yaml
https://github.com/pires/kubernetes-elasticsearch-cluster
镜像到这个地址: https://github.com/lannyMa/kubernetes-elasticsearch-cluster内含elk 普罗 harbor的yaml,比较实用
https://github.com/cnych/k8s-repo

集群yaml里搜HTTP_ENABLE,统一打开head访问权限

es-master.yaml

        - name: HTTP_ENABLEvalue: "true"- name: HTTP_CORS_ALLOW_ORIGINvalue: "*"

创建集群

创建pv

$ cat es-data_claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:name: opspvcnamespace: kube-opsannotations:volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:accessModes:- ReadWriteManyresources:requests:storage: 1Gi

先创建svc类型

后依次创建 data master client kibana

访问es api

访问kibana

打开kibana,插入示例数据

PUT _bulk
{"index":{"_index":"lagou","_type":"job","_id":"1"}}
{"title":"python分布式爬虫开发","salary_min":15000,"city":"深圳","company":{"name":"腾讯","company_addr":"深圳市软件园"},"publish_date":"2017-11-11","comments":15}
{"index":{"_index":"lagou","_type":"job","_id":"2"}}
{"title":"django开发","salary_min":15000,"city":"上海","company":{"name":"阿里","company_addr":"广州市软件园"},"publish_date":"2017-11-12","comments":20}

打开head查看数据

 docker run -d -v /etc/localtime:/etc/localtime --restart=always -p 9100:9100 mobz/elasticsearch-head:5

查看数据

todo:Curator(馆长) 管理索引的工具

Curator的主要用途:

• 获取索引列表

• 过滤索引,删除/快照

• 可以对索引做这些操作

elk整体链条架构

es版本兼容

    "minimum_wire_compatibility_version" : "5.6.0","minimum_index_compatibility_version" : "5.0.0"

curl http://10.100.32.137:9200{"name" : "es-data-5c5969967-wb2b8","cluster_name" : "myesdb","cluster_uuid" : "qSps-b9dRI2ngGHBguJ44Q","version" : {"number" : "6.3.0","build_flavor" : "default","build_type" : "tar","build_hash" : "424e937","build_date" : "2018-06-11T23:38:03.357887Z","build_snapshot" : false,"lucene_version" : "7.3.1","minimum_wire_compatibility_version" : "5.6.0","minimum_index_compatibility_version" : "5.0.0"},"tagline" : "You Know, for Search"
}