开发打印出来的日志里带有完整的手机号码，收集到elk里，这属于用户的敏感信息，所以理应把手机号码中的某部分用***代替。

没有直接在生产环境去试验，于是用docker部署一个elk环境出来试验，docker的elk，在https://github.com/deviantony/docker-elk，安装部署，参考：https://blog.csdn.net/Thinking771470736/article/details/107066060

像文章里一样，一开始我也打算用nc来调试。下面就遇到一些问题

1 端口不通

[root@localhost docker-elk]# echo "nc-test" | nc -u 192.168.245.4 8010

read(net): Connection refused

既然端口不通，好的，那就马上想到装telnet来测试，结果在logstash里想yum install -y telnet安装一下telnet

2 在docker里yum安装软件遇到权限问题

[root@localhost pipeline]# dexec docker-elk_logstash_1 bash

bash-4.2$ yum install -y telnet

Loaded plugins: fastestmirror, ovl

ovl: Error while doing RPMdb copy-up:

[Errno 13] Permission denied: '/var/lib/rpm/.dbenv.lock'

You need to be root to perform this command.

bash-4.2$

明显就是权限问题，那么，docker跑起来时，怎么用root身份进去呢？

3 docker用户身份，可以在docker run时指定

docker run -itd --user root ..... .... ....

由于用的是docker-compose，docker-compse.yml指定的方式为，以下是某部分，注意user: root

logstash:

build:

context: logstash/

args:

ELK_VERSION: $ELK_VERSION

volumes:

- type: bind

source: ./logstash/config/logstash.yml

target: /usr/share/logstash/config/logstash.yml

read_only: true

- type: bind

source: ./logstash/pipeline

target: /usr/share/logstash/pipeline

read_only: true

- type: bind

source: ./logstash/logs

target: /tmp

ports:

- "5044:5044"

- "5000:5000/tcp"

- "5000:5000/udp"

- "8010:8010"

- "8010:8010/udp"

- "9600:9600"

user: root

environment:

4 nc的安装

这些问题都搞定后，接下来安装nc来做测试，我是通过yum install nc -y安装的，装好后一试，就遇到

Ncat: Connection refused的问题，网友说：

在CentOS7.X中使用yum install -y nc安装的nc实际安装的是nmap-ncat(ncat命令),但ncat这个命令没有端口扫描功能，但为何在系统中又可以使用nc命令呢，归根于软链接文件

文章在：https://www.itbkz.com/11199.html

解决方案是自己源码安装nc，这里就不详细说

这些都弄好后，接下来就与logstash的正则表达式相关了，关于这部分的知识，找了一堆文章，有兴趣的可以参考下

https://doc.yonyoucloud.com/doc/logstash-best-practice-cn/filter/grok.html

https://doc.yonyoucloud.com/doc/logstash-best-practice-cn/filter/mutate.html

https://anbc.gitbooks.io/elk-handbook/content/81grokzheng_ze_guo_lv_qi_pei_zhi.html

https://www.jianshu.com/p/5df5055070b2

https://regex101.com/r/m0aoOx/1

https://www.cnblogs.com/sparkdev/p/10606810.html

http://grokdebug.herokuapp.com/

https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub

https://www.cnblogs.com/caoweixiong/p/12579498.html

https://www.cnblogs.com/sparkdev/p/10606810.html

https://mmx362003.gitbooks.io/elk-stack-guide/content/logstash_grok.html

mutate {

gsub => ["message", "(\d{6})(\d{6})(\d{4})", "\1######\3"]

add_tag => "Masked CardNo"

}

手机号的

## Add your filters / logstash plugins configuration here

filter {

mutate {

gsub => [

"phone", "(\d{3})(\d{4})(\d{4})", "\1***\3"

]

}

}

phone是我们json里的一个字段，也可以全匹配message，看需要

## Add your filters / logstash plugins configuration here

filter {

mutate {

gsub => [

"message", "(\d{3})(\d{4})(\d{4})", "\1***\3"

]

}

}

另外，filter/mutate 内部执行次序是这样的

rename(event) if @rename

update(event) if [@update]

replace(event) if @replace

convert(event) if @convert

gsub(event) if @gsub

uppercase(event) if @uppercase

lowercase(event) if @lowercase

strip(event) if @strip

remove(event) if @remove

split(event) if @split

join(event) if @join

merge(event) if @merge

filter_matched(event)

到此为止