linux去重某一列,Linux shell命令统计某列去重后的值
需求:根据mac获取,mac对应的硬件版本和软件版本。在根据硬件版本统计mac的数量
$ head test_1.txt
00:07:67,ec2108,hwv010537p0000
00:07:63,z83,znv861010p1225
00:07:67,ec6108v9u_ca_sccdx,hwv218012p0000
00:07:63,sbox8900hisi3716v3,zgv865012p0001
00:07:67,ec6108v9u_ca_sccdx,hwv218012p0000
00:21:26,ec6108v9u_ca_sccdx,hwv218012p0000
00:07:67,ec6108v9u_ca_sccdx,hwv218012p0000
00:07:63,stbhd-st7105-000,ztv70433089p0002
00:21:26,ec6108v9u_ca_sccdx,hwv215012p0000
00:21:26,ec6108v9u_ca_sccdx,hwv218012p0000
第二列去重
$ cat test_2.txt | awk -f ',' '{print $2}' | sort | uniq
ec2108
ec6108v9u_ca_sccdx
sbox8900hisi3716v3
stbhd-st7105-000
z83
将字符串转化为列表
import os
path = "c:\\users\\administrator\\desktop\\stb\\dd.txt"
read = open(path,"r")
list_ = []
for line in read:
line_ = line.strip()
list_.append(line_)
print(list_)
根据厂商和硬件版本获取对应的软件版本
方法一:
get gather-012-20180109/_search
{
"size": 0,
"aggs": {
"1": {
"terms": {
"field": "dev_hard_ver",
"size": 200
},
"aggs": {
"2": {
"terms": {
"field": "soft_ver",
"size": 1
}
}
}
}
}
}
方法二:
import os
from elasticsearch import elasticsearch
path = "c:\\users\\administrator\\desktop\\stb\\fact_stb.txt"
write_file = open(path,"a+")
es = elasticsearch()
def get_data(dev_hard_ver,dev_vendor_name):
index_ = "gather-012-20180109"
try:
rs = es.search(index=index_,body={
"size": 1,
"query": {
"bool": {
"must": [
{"term": {
"dev_hard_ver": {
"value": dev_hard_ver
}
}
},
{
"term": {
"dev_vendor_name": {
"value": dev_vendor_name
}
}
}
]
}
},
"_source": ["soft_ver","dev_vendor_name","dev_hard_ver"]
})
return rs
except:
return -1
dev_hard_ver=["02l0b61701mtas0001", "0x209", "0x441", "0x485", "0xe0a", "0xe17", "16060102l0b61701mtas0001", "1828", "1903", "2035", "5202", "99007002l0b6170635100791", "9900b002l0b61701iho3000t", "990104010004102300000001", "99010402l0b6083100000001", "990501010003170908a5c800", "a7.32.64.a0", "b860av2.1", "bjds2z-52", "chep3000sc", "chi619112", "chiho3000", "chiho3000a", "chiho3000jx2", "chiho33", "chiho3300a", "chiho3300ad", "cub-androidstb", "ec2106v1", "ec2108", "ec6108v9_pub_gdydx", "ec6108v9u_ca_sccdx", "ec6108v9u_pub_hbjdx", "ec6108v9u_pub_sccdx", "g2-40_2544", "hg510i", "hg650-tdswh", "hg650-tdswh-11", "ht670-v", "hy_v1.0", "hy107", "hy112", "hy127", "hy2021", "hy34e71c02l0b6v003hg510p04", "hy34e71c02l0b6v3.0hg510p10", "hy44170102l0b6v003hg510p04", "hy9900e0013798kk0120170001", "hy990104010004102300000001", "hyc37.e", "hychep920sc", "hychep921sc", "hyee", "hyhg510iv1.0", "hyhg510pv3.0", "hyty_h1.00.00", "hyv10-androidigw-068", "hyyix-g210v1.1", "ibhn-androidstb-068", "ibhr-androidstbga-069", "ibhu-androidstb-068", "ibhx-androidstbga-068", "ibhx-androidstbga-069", "iptv-s63-v1.1", "iptv-s65-v1.1", "iptv-s65-v1.2", "itv628 hd", "kb1.100.01h", "m8043v02", "mr222-dsgh-19", "mx_ca_sccdx", "s80-e-r1.01.01", "sbox8900hisi3716v3", "sbox8900hisi3716v386501007", "stbab-androidstb-068", "stbab-androidstbga-069", "stbhd-aml8626x-000", "stbhd-st7105-000", "stbhd-st7105-128-256-000", "stbho-aml8626x-000", "tgh0001", "ty_a1.00.00", "ty_h1.00.00", "v1", "v10-androidigw-068", "v2.1", "z82", "z83", "z85e", "zxb700v5c(ibhl-000-000)", "zxb860a(ibhr-androidstb-068)", "zxb860a(ibhr-androidstbga-068)"]
dev_vendor_name=["scty", "changhong", "jiuzhou", "jiuzhou", "hisense", "hisense", "scty", "skyworth", "skyworth", "skyworth", "huawei", "zte", "changhong", "skyworth", "skyworth", "zhaoneng", "utstarcom", "zte", "fiberhome", "changhong", "changhong", "changhong", "changhong", "changhong", "changhong", "changhong", "changhong", "youhua", "huawei", "huawei", "huawei", "huawei", "huawei", "huawei", "yinhe", "utstarcom", "fiberhome", "fiberhome", "zhaoneng", "scty", "hisense", "hisense", "hisense", "skyworth", "yhtc", "yhtc", "yhtc", "fiberhome", "skyworth", "huawei", "changhong", "changhong", "hisense", "utstarcom", "yhtc", "scty", "zte", "ccs", "zte", "zte", "zte", "zte", "zte", "chinagci", "chinagci", "chinagci", "changhong", "30kt", "huawei", "fiberhome", "huawei", "chinagci", "sunniwell", "sunniwell", "zte", "zte", "zte", "zte", "zte", "zte", "yihekeji", "scty", "scty", "zte", "zte", "fiberhome", "zhaoneng", "zhaoneng", "zhaoneng", "zte", "zte", "zte"]
for i in range(len(dev_vendor_name)):
rs = get_data(dev_hard_ver[i],dev_vendor_name[i])
if rs!=-1:
for hit in rs['hits']['hits']:
soft_ver=hit['_source']['soft_ver']
if soft_ver is none:
soft_ver='none'
write_file.write(soft_ver+"\n")
else:
write_file.write("-1" + "\n")
write_file.close()
总结
以上就是这篇文章的全部内容了,希望本文的内容对大家的学习或者工作具有一定的参考学习价值,谢谢大家对萬仟网的支持。如果你想了解更多相关内容请查看下面相关链接
linux去重某一列,Linux shell命令统计某列去重后的值相关推荐
- linux列去重命令,Linux shell命令统计某列去重后的值
需求:根据MAC获取,MAC对应的硬件版本和软件版本.在根据硬件版本统计MAC的数量 $ head test_1.txt 00:07:67,EC2108,HWV010537P0000 00:07:63 ...
- 操作系统必知必会:Linux(1)常用的Shell命令
操作系统必知必会:Linux(1)常用的Shell命令 常用的Shell命令 磁盘管理 cd df du mkdir pwd ls 文件管理 cat chmod chown cp diff file ...
- linux 终端 画圆,16个圆桌面Linux用户必须要知道的Shell命令
16个圆桌面Linux用户必须要知道的Shell命令 16个圆桌面Linux用户必须要知道的Shell命令 日期:2014-05-16 浏览次数:20305 次 16个桌面Linux用户必须要知道的S ...
- linux 查看桌面目录下,16个桌面Linux用户必须要知道的Shell命令
有些人仍然会有这中愚蠢的想法,他们认为使用Linux就必须使用Linux shell命令.胡说!你可以不懂得任何Linux命令,比如说ps,grep,ls等,但是你仍然可以使用很多现代的Linux桌面 ...
- linux shell两条命令,Linux最常用的18个Shell命令
有些人仍然会有这中愚蠢的想法,他们认为使用Linux就必须使用Linux shell命令.胡说!你可以不懂得任何Linux命令,比如说ps,grep,ls等,但是你仍然可以使用很多现代的Linux桌面 ...
- linux shell 桌面,16个桌面Linux用户必须要知道的Shell命令
有些人仍然会有这中愚蠢的想法,他们认为使用就必须使用Linux命令.胡说!你可以不懂得任何Linux命令,比如说ps,grep,ls等,但是你仍然可以使用很多现代的Linux桌面发行版. 5D9p:B ...
- HBase shell 命令没有输完换行后结束本次输入重新输入命令
HBase shell 命令没有输完换行后结束本次输入重新输入命令 前言 在使用hbase shell命令行时,命令还没有输完或者在还没有输入右边单引号时换行后怎么结束本次输入重新输入. 问题如下: ...
- Ubuntu中shell命令-(1)-shell是什么?-Linux内核有哪些东西?-shell命令解释器包含哪几个种类?
第一,shell是什么? linux系统的shell类似于window下的cmd.是一种命令解析器,用来和linux内核沟通.接收用户命令,调用相应的应用程序(内核应用程序) 第二,扩展:linux内 ...
- linux pssh parallel-ssh 批量执行远程shell命令
目录 pssh简介 pssh参数列表 使用实例 pssh简介 文档地址: https://parallel-ssh.readthedocs.io/en/latest/ python异步并行SSH客户端 ...
最新文章
- python使用fpdf生成结账发票pdf文件
- 算法-判断是否是完全二叉树
- Java:网络编程值TCP的使用
- python方向键键值_Python实现的字典排序操作示例【按键名key与键值value排序】
- 算法分析 运动员循环赛_「98跑」大众跑者的训练比专业运动员更累!
- 批处理之字符串处理和数值计算
- c#: 任务栏进度显示(TaskbarManager)
- [转载] python中字典中追加_python 中字典中的删除,pop 方法与 popitem 方法
- IE发现新的零日攻击漏洞 用户可采取缓解措施
- ai人工智能对话了_对话式AI:智能虚拟助手和未来之路。
- vray渲染里服务器信息,Vray使用手册——设置V-Ray渲染服务器
- 基于java的企业进销存管理系统
- cdr2018更新内容
- 应对反爬如何爬取拉勾网信息(非常详细)
- English-Phonics
- 吴式太极拳的特点-和基础要求
- windows云服务器,如何使用windows云服务器
- HTML 笔记/案例
- OpenCV 文字绘制----cv::putText详解
- 中国信号调理器行业市场供需与战略研究报告
热门文章
- 第三篇:POSIX标准中的 “ 限制 ”
- Codeforces Round #250 (Div. 2)—A. The Child and Homework
- 制作MACOSX10.10.3/10.9安装启动盘U盘的教程
- Entropay(欧贝通)
- GetManifestResourceStream得到的Stream是null的解决
- 转: ADO Connection Strings
- dw 网页 html 布局,Dreamweaver网页制作之CSS布局规则
- 问题 A: 阿正的忐忑不安(C语言)
- 信息学奥赛一本通 1167:再求f(x,n)
- 信息学奥赛一本通 2004:【20CSPJ普及组】优秀的拆分 | 洛谷 P7071 [CSP-J2020] 优秀的拆分