1.概述

转载：https://my.oschina.net/waterbear/blog/4493422

主要是为了实操一遍，同时更改curl的方式。

前提

操作系统：mac
ElasticSearch 7.8

创建索引


PUT rcmdPUT /rcmd/_mapping
{"properties": {"tags": {"type": "keyword","store": true},"update_time": {"type": "date","store": true}}
}

两个字段：

tags，文章的兴趣标签，keyword类型就是不需要全文检索，标签以数组的形式存放
update_time，更新时间，这是给兴趣推荐加一个额外的排序条件，实际项目中往往是需要结合时间和匹配度来排序的

模拟数据


POST /rcmd/_doc
{"tags": ["布料","抹布","裤子","衣服","生活"],"update_time": "2020-06-01T00:02:11.030"}# 再插入一条，同样标签，但是时间不一样，后面例子中有妙用POST /rcmd/_doc{"tags": ["布料","抹布","裤子","衣服","生活"],"update_time": "2020-07-01T00:02:11.030"}GET /rcmd/_searchPOST /rcmd/_doc
{"tags": ["啤酒","米酒","饮料","餐饮","生活"],"update_time": "2020-06-02T00:02:11.030"
}POST /rcmd/_doc
{"tags": ["火锅","自助餐","外卖","烧烤","餐饮"],"update_time": "2020-06-03T00:02:11.030"}POST /rcmd/_doc
{"tags": ["太阳","月亮","大海","星星","自然"],"update_time": "2020-06-01T00:02:11.030"}POST /rcmd/_doc
{"tags": ["人类","动物","植物","地球","自然"],"update_time": "2020-06-01T00:02:11.030"}POST /rcmd/_doc
{"tags": ["男人","女人","小孩","老人","人类"],"update_time": "2020-06-02T00:02:11.030"}

最终数据如下

固定分数查询

GET /rcmd/_search {"query": {"bool": {"should": [{"constant_score": {"boost": 1,"filter": {"match": {"tags": "生活"}}}},{"constant_score": {"boost": 1,"filter": {"match": {"tags": "衣服"}}}},{"constant_score": {"boost": 1,"filter": {"match": {"tags": "火锅"}}}}]}}}

should表达式的意义是匹配“生活”、“衣服”、“火锅”三个标签中任何一个的文章都可以返回。用constant_score查询，如果某个文章涵盖标签越多分值就越高。也就是说如果某个文章标签完全涵盖了这三个标签，那么它的分值最高的。查询结果如下：

{"took": 2,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": {"value": 4,"relation": "eq"},"max_score": 2.0,"hits": [{"_index": "rcmd","_type": "_doc","_id": "brQO63MBTdXKc2eArv9A","_score": 2.0,"_source": {"tags": ["布料","抹布","裤子","衣服","生活"],"update_time": "2020-06-01T00:02:11.030"}},{"_index": "rcmd","_type": "_doc","_id": "b7QP63MBTdXKc2eAPf_Y","_score": 2.0,"_source": {"tags": ["布料","抹布","裤子","衣服","生活"],"update_time": "2020-07-01T00:02:11.030"}},{"_index": "rcmd","_type": "_doc","_id": "cLQQ63MBTdXKc2eA6_8v","_score": 1.0,"_source": {"tags": ["啤酒","米酒","饮料","餐饮","生活"],"update_time": "2020-06-02T00:02:11.030"}},{"_index": "rcmd","_type": "_doc","_id": "cbQS63MBTdXKc2eAcP-N","_score": 1.0,"_source": {"tags": ["火锅","自助餐","外卖","烧烤","餐饮"],"update_time": "2020-06-03T00:02:11.030"}}]}
}

有两篇文章涵盖了其中两个标签“生活”和“衣服”，得分为2，排到了前面。这个排序基本满足了兴趣匹配的要求。

兴趣标签权值

实际的项目中往往是用户的兴趣标签的权值不一样，假设用户的兴趣标签是[“火锅”,“生活”,“衣服”]，排在越前面的权重越高，查询的时候需要给关键词设定权重，上面的查询语句所有boost都是默认值1，现在根据需求改动权值再查询。

GET /rcmd/_search
{"query": {"bool": {"should": [{"constant_score": {"boost": 1,"filter": {"match": {"tags": "生活"}}}},{"constant_score": {"boost": 4,"filter": {"match": {"tags": "衣服"}}}},{"constant_score": {"boost": 6,"filter": {"match": {"tags": "火锅"}}}}]}}}

分别给三个词加上权重6、4、1，查询结果如下：

{"took": 1,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": {"value": 4,"relation": "eq"},"max_score": 6.0,"hits": [{"_index": "rcmd","_type": "_doc","_id": "cbQS63MBTdXKc2eAcP-N","_score": 6.0,"_source": {"tags": ["火锅","自助餐","外卖","烧烤","餐饮"],"update_time": "2020-06-03T00:02:11.030"}},{"_index": "rcmd","_type": "_doc","_id": "brQO63MBTdXKc2eArv9A","_score": 5.0,"_source": {"tags": ["布料","抹布","裤子","衣服","生活"],"update_time": "2020-06-01T00:02:11.030"}},{"_index": "rcmd","_type": "_doc","_id": "b7QP63MBTdXKc2eAPf_Y","_score": 5.0,"_source": {"tags": ["布料","抹布","裤子","衣服","生活"],"update_time": "2020-07-01T00:02:11.030"}},{"_index": "rcmd","_type": "_doc","_id": "cLQQ63MBTdXKc2eA6_8v","_score": 1.0,"_source": {"tags": ["啤酒","米酒","饮料","餐饮","生活"],"update_time": "2020-06-02T00:02:11.030"}}]}
}

可以看到包含“火锅”的文章排到了第一，包含“衣服”和“生活”的文章虽然两个词都命中，但是在权值的弱化之下排到了第二第三位。

多条件排序


GET /rcmd/_search
{"query": {"function_score": {"query": {"bool": {"must": [{"range": {"update_time": {"from": "2020-06-01","to": "2020-08-01"}}},{"bool": {"should": [{"term": {"tags": {"term": "火锅","boost": 2}}},{"term": {"tags": {"term": "衣服","boost": 1}}},{"term": {"tags": {"term": "生活","boost": 1}}}]}}]}},"functions": [{"gauss": {"update_time": {"scale": "3d","origin": "2020-07-02T00:01:00.000"}}}]}},"_source": {"include": ["tags","update_time"]},"from": 0,"size": 10
}

以上是相对完整的一个查询，首先对update_time发布时间做了限制，只选择一定范围内的数据，随后是标签的匹配，多个标签匹配条件之间是"OR"的关系，标签具有不同的权重，接下来用衰减函数gauss对update_time做衰减排序，衰减函数的意义是越近越好，scale": "3d"就是以3天为一个阶梯先对数据进行排序，相同阶梯内的数据再按照标签匹配度排序。注：gauss中的origin可以不指定最终的查询结果：

{"took": 2,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": {"value": 4,"relation": "eq"},"max_score": 3.6649413,"hits": [{"_index": "rcmd","_type": "_doc","_id": "b7QP63MBTdXKc2eAPf_Y","_score": 3.6649413,"_source": {"update_time": "2020-07-01T00:02:11.030","tags": ["布料","抹布","裤子","衣服","生活"]}},{"_index": "rcmd","_type": "_doc","_id": "cbQS63MBTdXKc2eAcP-N","_score": 4.4511746E-28,"_source": {"update_time": "2020-06-03T00:02:11.030","tags": ["火锅","自助餐","外卖","烧烤","餐饮"]}},{"_index": "rcmd","_type": "_doc","_id": "cLQQ63MBTdXKc2eA6_8v","_score": 1.764942E-30,"_source": {"update_time": "2020-06-02T00:02:11.030","tags": ["啤酒","米酒","饮料","餐饮","生活"]}},{"_index": "rcmd","_type": "_doc","_id": "brQO63MBTdXKc2eArv9A","_score": 2.8566082E-32,"_source": {"update_time": "2020-06-01T00:02:11.030","tags": ["布料","抹布","裤子","衣服","生活"]}}]}
}

同样是匹配了“衣服”和“生活”的两篇文章，一篇在最前面，一篇在最后面，是因为update_time的缘故，一篇是7月1日发布的，另一篇在6月1日，不在同一时间阶梯内，日期久远的排到了后面。中间的两篇，各自匹配了一个标签，分别是“烧烤”和“生活”，两篇文章时间阶梯没有明显的区别，然而匹配“火锅”的排到了前面，是因为“火锅”的关键词加了较高的权重。至此，我们实现了按照标签匹配文章，并且结合了时间因素和匹配度评分的兴趣推荐。

后续问题

本文仅仅实现了推荐中信息的匹配和排序，实际的推荐系统中还有信息流中过滤已推文章的问题，取备选文章和历史文章的交集是比较耗时的运算；此外还要解决用户Feed流推拉问题。
在本文中没有提及用户兴趣标签的累积操作，理论上来说只要用户读了相应的文章，就根据文章的标签给用户兴趣标签累计加分，实际项目中往往需要处理兴趣标签截断统一降权，例如标签库有1万个标签，用户进行多次阅读行为之后，一个用户和1万个标签都会有关系，分值大小不同的区别而已，如果一直根据高分的兴趣标签给用户推荐文章，兴趣标签就形成了马太效应，新的兴趣标签没有机会超过累积高分的标签，兴趣推送会越来越窄，这时候就需要截取一定数量的兴趣标签，例如截取前100个，然后统一降分，给用户的新兴趣超赶机会。
以上例子没有在超大数据环境下测试过，还没有具体的性能指标。

【Elasticsearch】用ElasticSearch实现基于标签的兴趣推荐相关推荐

TP5 实现基于标签简单的推荐算法
1.算法思想 1.1.理解算法过程我们在写算法的时候要先理解我们的对象和之间的关系,我这里举例供求信息和用户设置标签,两者关系是,系统会根据用户设置的标签来匹配与其相似度较高的,同时用户发布的供求信 ...
《推荐系统笔记（十二）》聚类生成标签以及基于标签的TopN推荐
如何生成标签? 一般有 PGC(专家生成) UGC(用户生成) 但是,面对海量的数据需要打标签时,人工打标签太过昂贵和耗时,一个可行方法是对数据使用聚类,然后将聚类结果作为标签使用. 下面给一个聚类的 ...
基于标签的实时短视频推荐系统 | 深度
作者 | gongyouliu 转载自大数据与人工智能(ID: ai-big-data) 导语:作者在<基于内容的推荐算法>这篇文章中对基于内容的推荐算法做了比较详细的讲解,其中一类非常重 ...
基于标签的实时短视频推荐系统
点击上方"大数据与人工智能","星标或置顶公众号" 第一时间获取好内容作者丨gongyouliu 这是作者的第14篇文章,约1.1万字,阅读需70分钟以下为 ...
基于Vue的兴趣活动推荐APP的设计与实现_kaic
摘要近年来,随着互联网不断的普及与发展,生活也变的多姿多彩,每个人几乎都有属于自己的小小兴趣圈子,但人们的兴趣却不止一个,大多数人是没有时间为自己的兴趣"买单"的.为了解决 ...
推荐系统7—基于标签的推荐系统
本章将采用两个不同的数据集评测基于标签的物品推荐算法.一个是Delicious数据集,另一个是CiteULike数据集.Delicious数据集中包含用户对网页的标签记录.它每一行由4部分组成,即时间 ...
Elasticsearch单机版安装（基于CentOS7）
CentOS7下Elasticsearch单机版安装 1.ElasticSearch概述 ElasticSearch是一款基于Apache Lucene构建的开源搜索引擎,它采用Java编写并使用Lu ...
【Elasticsearch】Elasticsearch自定义评分的N种方法
1.概述首先参考文章:[Elasticsearch]Elasticsearch 相关度评分 TF&IDF 然后转载文章:实战 | Elasticsearch自定义评分的N种方法 2.三个问题 ...
标签 href 怎么拼接_【微信】用户-标签的兴趣建模
这一篇分享的是CIKM2020微信的learning to build user-tag profile,主要介绍了微信看一看("Top Stories")中,如何进行用户-标签的 ...

【Elasticsearch】用ElasticSearch实现基于标签的兴趣推荐