elk logstach收集交换机日志

前提：ELK环境已经安装完成，具体操作查看另外篇文章

一、交换机配置

添加：info-center loghost 192.168.2.123，IP地址是logstash服务器，默认是UDP514端口发送数据

<SW46>display  version
Huawei Versatile Routing Platform Software
VRP (R) software, Version 5.70 (S2700 V100R006C05)
Copyright (C) 2003-2013 HUAWEI TECH CO., LTD
Quidway S2700-18TP-EI-AC Routing Switch uptime is 5 weeks, 2 days, 0 hour, 35 minutesEDFE 0(Master) : uptime is 5 weeks, 2 days, 0 hour, 35 minutes
64M bytes DDR Memory
16M bytes FLASH
Pcb      Version :  VER C
Basic  BOOTROM  Version :  149 Compiled at Mar 15 2013, 11:02:25
Software Version : VRP (R) Software, Version 5.70 (V100R006C05)
<SW46>display cu | in infoinfo-center loghost 192.168.2.123snmp-agent sys-info version all

二、logstash配置

1、关闭rsyslog服务，因为这个会占用514端口

[root@node1 ~]# systemctl stop rsyslog
[root@node1 ~]# systemctl status rsyslog
● rsyslog.service - System Logging ServiceLoaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)Active: inactive (dead) since Mon 2019-07-08 01:49:41 EDT; 1 day 23h agoDocs: man:rsyslogd(8)http://www.rsyslog.com/doc/Process: 4696 ExecStart=/usr/sbin/rsyslogd -n $SYSLOGD_OPTIONS (code=exited, status=0/SUCCESS)Main PID: 4696 (code=exited, status=0/SUCCESS)

2、设置logstash用root账户启动
发现华为的设备在指定syslog的时候没有办法自定义端口，就只好用默认的端口514，就会产生一个问题，1024以下的端口需要root用户才能用。就简单暴力的使用root启动logstash（修改User和Group）

[root@node1 ~]# cat /etc/systemd/system/logstash.service
[Unit]
Description=logstash[Service]
Type=simple
User=root
Group=root
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384[Install]
WantedBy=multi-user.target

3、编辑logstash配置文件，根据监听交换机端口区分不通网络设备型号(直接复制可用，修改下IP地址)

[root@node1 ~]# vim /etc/logstash/conf.d/switch.conf
[root@node1 ~]# cat  /etc/logstash/conf.d/switch.conf
input{tcp { port => 5002 type => "Cisco"}udp { port => 514type => "HUAWEI"}udp { port => 5002type => "Cisco"}udp { port => 5003type => "H3C"}
}
filter {if [type] == "Cisco" {grok {match => { "message" => "<%{BASE10NUM:syslog_pri}>%{NUMBER:log_sequence}: .%{SYSLOGTIMESTAMP:timestamp}: %%{DATA:facility}-%{POSINT:severity}-%{CISCO_REASON:mnemonic}: %{GREEDYDATA:message}" }match => { "message" => "<%{BASE10NUM:syslog_pri}>%{NUMBER:log_sequence}: %{SYSLOGTIMESTAMP:timestamp}: %%{DATA:facility}-%{POSINT:severity}-%{CISCO_REASON:mnemonic}: %{GREEDYDATA:message}" }add_field => {"severity_code" => "%{severity}"}overwrite => ["message"]}
}elseif [type] == "H3C" {grok {match => { "message" => "<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{YEAR:year} %{DATA:hostname} %%%{DATA:vvmodule}/%{POSINT:severity}/%{DATA:digest}: %{GREEDYDATA:message}" }remove_field => [ "year" ]add_field => {"severity_code" => "%{severity}"}overwrite => ["message"]}
}elseif [type] == "HUAWEI" {grok {match => { "message" => "<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{DATA:hostname} %%%{DATA:ddModuleName}/%{POSINT:severity}/%{DATA:Brief}:%{GREEDYDATA:message}"}match => { "message" => "<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{DATA:hostname} %{DATA:ddModuleName}/%{POSINT:severity}/%{DATA:Brief}:%{GREEDYDATA:message}"}remove_field => [ "timestamp" ]add_field => {"severity_code" => "%{severity}"}overwrite => ["message"]}
}
#mutate {
#        gsub => [
#        "severity", "0", "Emergency",
#        "severity", "1", "Alert",
#        "severity", "2", "Critical",
#        "severity", "3", "Error",
#        "severity", "4", "Warning",
#        "severity", "5", "Notice",
#        "severity", "6", "Informational",
#        "severity", "7", "Debug"
#        ]
#    }
}
output{stdout {
#将日志输出到当前终端上显示codec => rubydebug
}
#同时也发送到elasticsearchelasticsearch {index =>"syslog-%{+YYYY.MM.dd}"hosts => ["192.168.2.10:9200"]}
}

4、切换到logstash的bin目录，检测配置文件是否有错。显示OK则表示配置文章没有问题

[root@node1 ~]#cd /usr/share/logstash/bin/
[root@node1 bin]# ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/switch.conf --config.test_and_exit
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
Configuration OK

5、直接启动，查看到数据显示在终端

[root@node1 bin]# ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/switch.conf
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
{"@version" => "1","host" => "192.168.88.46","@timestamp" => 2019-07-10T05:23:49.727Z,"message" => "<188>Jul 10 2019 05:23:49 SW46 %%01SHELL/4/TELNETFAILED(l)[78]:Failed to login through telnet. (Ip=**, UserName=**, Times=1)","type" => "HUAWEI","tags" => [[0] "_grokparsefailure"]
}

6、ctrl+C结束，然后启动logstash服务

[root@node1 bin]# systemctl start logstash

可选：如果还是无法启动，试着修改用户权限，然后再重启服务

[root@node1 ~]# chown root /var/log/logstash/logstash-plain.log
[root@node1 ~]# chown -R root /var/lib/logstash/
[root@node1 ~]# systemctl restart logstash.service

7、查看端口是否监听

[root@node1 ~]# ss -ntlpu
Netid  State      Recv-Q Send-Q                                   Local Address:Port                                                  Peer Address:Port
udp    UNCONN     0      0                                                    *:68                                                               *:*                   users:(("dhclient",pid=4470,fd=6))
udp    UNCONN     0      0                                            127.0.0.1:323                                                              *:*                   users:(("chronyd",pid=4421,fd=1))
udp    UNCONN     0      0                                                    *:514                                                              *:*                   users:(("java",pid=21845,fd=72))
udp    UNCONN     0      0                                                    *:5002                                                             *:*                   users:(("java",pid=21845,fd=73))
udp    UNCONN     0      0                                                    *:5003                                                             *:*                   users:(("java",pid=21845,fd=74))
udp    UNCONN     0      0                                                  ::1:323                                                             :::*                   users:(("chronyd",pid=4421,fd=2))
tcp    LISTEN     0      128                                                  *:22                                                               *:*                   users:(("sshd",pid=4691,fd=3))
tcp    LISTEN     0      100                                          127.0.0.1:25                                                               *:*                   users:(("master",pid=5020,fd=13))
tcp    LISTEN     0      128                                                 :::5002                                                            :::*                   users:(("java",pid=21845,fd=71))
tcp    LISTEN     0      128                                                 :::9200                                                            :::*                   users:(("java",pid=14705,fd=162))
tcp    LISTEN     0      128                                                 :::9300                                                            :::*                   users:(("java",pid=14705,fd=105))
tcp    LISTEN     0      128                                                 :::22                                                              :::*                   users:(("sshd",pid=4691,fd=4))
tcp    LISTEN     0      100                                                ::1:25                                                              :::*                   users:(("master",pid=5020,fd=14))
tcp    LISTEN     0      50                                ::ffff:192.168.14.37:9600                                                            :::*                   users:(("java",pid=21845,fd=43))

三、回到elasticsearch查看索引，能看到syslog-*相关日志

[root@master ~]# curl '192.168.2.10:9200/_cat/indices?v'
health status index                       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .kibana                     kSyxiC6-RdmJmp8StSdbYg   1   1          3            0     37.3kb         18.6kb
green  open   syslog-2019.07.10           xAhv7gfOQFOCMsqzW5juAA   5   1        104            0    812.1kb          406kb

四、kibana查看交换机日志

1、添加索引

2、查看交换机收集日志

elk logstach收集交换机日志相关推荐

华为交换机S5700配置syslog收集交换机日志
转载至:[网络-实验]华为交换机S5700配置syslog收集交换机日志_猫先生的早茶的博客-CSDN博客_华为交换机收集日志拓补说明某企业网络拓补如图交换机配置vlanif1的IP地址为192 ...
centos6.5下安装配置ELK及收集nginx日志
Elasticsearch 是个开源分布式搜索引擎,它的特点有:分布式,零配置,自动发现,索引自动分片,索引副本机制,restful风格接口,多数据源,自动搜索负载等. Logstash 是一个完全开 ...
ELK之收集haproxy日志
由于HAProxy的运行信息不写入日志文件,但它依赖于标准的系统日志协议将日志发送到远程服务器(通常位于同一系统上),所以需要借助rsyslog来收集haproxy的日志.haproxy代理nginx ...
Loki 收集Nginx日志以 grafana 可视化展示
背景通常用ELK来收集Nginx日志的,对于服务器较少的用elk则显得太重了,可以用loki+Promtail+grafana 代替. Loki类似elasticsearch,用于存储:Promta ...
ELK：收集k8s容器日志最佳实践
简介关于日志收集这个主题,这已经是第三篇了,为什么一再研究这个课题,因为这个课题实在太重要,而当今优秀的开源解决方案还不是很明朗: 就docker微服务化而言,研发有需求标准输出,也有需求文件输出, ...
ELK日志系统之使用Rsyslog快速方便的收集Nginx日志
常规的日志收集方案中Client端都需要额外安装一个Agent来收集日志,例如logstash.filebeat等,额外的程序也就意味着环境的复杂,资源的占用,有没有一种方式是不需要额外安装程序就能实 ...
配置使用rsyslog+loganalyzer收集防火墙及交换机日志
1.目的背景日志功能对于操作系统是相当重要的,在使用中,无论是系统还是应用等等,出了任何问题,我们首先想到的便是分析日志,查找问题原因. 自 CentOS 6 开始,我们的 CentOS 便开始使用 ...
ELK收集tomcat日志
转载来源 :ELK收集tomcat日志 :https://www.jianshu.com/p/9f4ba87e4e15 1.安装tomcat 注意:需要提前安装好java环境 [root@db01 ~ ...
ELK收集java日志
转载来源:ELK收集java日志 https://www.jianshu.com/p/957e8ead3f8d 由于elasticsearch本身产生的日志就是java日志,所以我们可以通过ELK平台 ...
ELK收集docker日志
转载来源 :ELK收集docker日志 : 1.安装docker ##安装依赖 yum install -y yum-utils device-mapper-persistent-data lvm2 ...