ELK实践（一）使用es搭建商品搜索项目

一、了解数据及建模

实验使用的数据是类Airbnb网站上的房屋信息，结构如下所示：

数据建模：
自定义分词器，以实现搜索时的自动补全功能，针对不同的字段定义不同的数据类型，以达到最优的效果：例如dynamic设置为false，对于不需要分词的字段直接设置类型为keyword，不需要检索的字段index设置为false等等。附上具体配置以供参考：

PUT testairbnb
{"settings": {"index": {"number_of_shards": 1,"number_of_replicas": 0,"analysis": {"analyzer": {"autosuggest_analyzer": {"filter": ["lowercase","asciifolding","autosuggest_filter"],"tokenizer": "standard","type": "custom"},"ngram_analyzer": {"filter": ["lowercase","asciifolding","ngram_filter"],"tokenizer": "standard","type": "custom"}},"filter": {"autosuggest_filter": {"max_gram": "20","min_gram": "1","token_chars": ["letter","digit","punctuation","symbol"],"type": "edge_ngram"},"ngram_filter": {"max_gram": "9","min_gram": "2","token_chars": ["letter","digit","punctuation","symbol"],"type": "ngram"}}}}},"mappings": {"doc": {"dynamic": false,"properties": {"accommodates": {"type": "integer"},"bathrooms": {"type": "integer"},"bed_type": {"type": "text","fields": {"keyword": {"type": "keyword","ignore_above": 256}}},"bedrooms": {"type": "integer"},"beds": {"type": "integer"},"date_from": {"type": "date","format": "yyyyMMdd"},"date_to": {"type": "date","format": "yyyyMMdd"},"has_availability": {"type": "boolean"},"host_image": {"type": "keyword","ignore_above": 256,"index": false},"host_name": {"type": "text","analyzer": "autosuggest_analyzer","fields": {"keyword": {"type": "keyword","ignore_above": 256}}},"image": {"type": "keyword","ignore_above": 256,"index":false},"listing_url": {"type": "keyword","ignore_above": 256},"location": {"type": "geo_point"},"name": {"type": "text","analyzer": "autosuggest_analyzer","fields": {"keyword": {"type": "keyword","ignore_above": 256}}},"price": {"type": "float"},"property_type": {"type": "text","fields": {"keyword": {"type": "keyword","ignore_above": 256}}},"room_type": {"type": "text","fields": {"keyword": {"type": "keyword","ignore_above": 256}}}}}}
}

二、导入数据

因为数据为csv格式，所有可以直接用csv插件将数据导入到es中。配置如下：

input{file {path => "/home/wfs/project/airbnb/airbnb.csv"sincedb_path => "/dev/null"start_position => "beginning"}
}filter{csv{columns => ["accommodates","bathrooms","bed_type","bedrooms","beds","date_from","date_o","date_rom","date_to","has_availability","host_image","host_name","image","listing_url","location","name","price","property_type","room_type"]}mutate{remove_field=>["message"]lowercase=>["has_availability"]}
}
output{elasticsearch{hosts => ["192.168.20.101:9200","192.168.20.102:9200"]index => "testairbnb"user => "elastic"password => "123456"}stdout{codec=>rubydebug}
}

使用lowercase将has_availability字段内容转换为小写

注意数据在导入es后，通过kibna查看会发现image和host_image字段还是以url的形式存在，我们需要做进一步的格式处理来实现以图片的形式存在的效果。

对image和host_image字段做如下处理：

对listing_url 字段做如下处理：

最后实现的效果如下所示：

然后就可以针对不同的条件做搜索查询了。

三、搭建搜索界面

现在就可以通过ReactiveSearch来搭建一个简易的搜索界面了
ReactiveSearch主页：https://opensource.appbase.io/reactive-manual/
github地址：https://github.com/appbaseio/reactivesearch
demo: https://github.com/appbaseio-apps/airbeds

安装：nodejs、yarn

安装依赖关系：
# yarn启动：默认3000端口
# yarn start

效果图：

这样就可以通过指明时间、价格等条件开进行搜索了。

配置说明：
从github上下载demo包：

git clone https://github.com/appbaseio-apps/airbeds

修改配置连接到es:

# cd airbeds/src/
# ls
App.css  App.js  App.test.js  index.css  index.js  logo.svg  registerServiceWorker.js

修改App.js文件，主要配置如下：

修改es配置文件：

# tail -3 elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: X-Requested-With, X-Auth-Token, Content-Type, Content-Length, Authorization, Access-Control-Allow-Headers, Accept
#修改es所有节点，然后重启服务

自定义组件：
ReactiveSearch主页上有对每个组件的详细说明及样例，可以直接使用，例如：

打包：
可以将调试好的项目进行打包yarn build，打包完成后将build放到任意有http服务的地方就可以运行。
package.json文件”homepage“设置为空
打包：yarn build
打包到build文件夹下，这样直接将build文件夹的文件放到httph或者nginx家目录下就可以直接80端口访问了。