CentOS部署ElasticSearch7.6.1集群
本文基于CentOS7.6,部署ElasticSearch7.6.1,詳細(xì)過程如下:
一、準(zhǔn)備工作
1、準(zhǔn)備機(jī)器
準(zhǔn)備3臺(tái)機(jī)器,安裝CentOS7.6。ip計(jì)劃如下:
192.168.2.11 es-master
192.168.2.12 es-node1
192.168.2.13 es-node2
2、關(guān)閉防火墻
systemctl stop firewalld systemctl disable firewalld3、安裝JDK
本處安裝jdk-8u231-linux-x64,并配置好環(huán)境變量。
4、禁用SELINUX
vim /etc/selinux/config將SELINUX設(shè)置為disabled:
SELINUX=disabled
5、修改主機(jī)名
#設(shè)置主機(jī)名
hostnamectl set-hostname es-node1通過hostname命令查看主機(jī)名。
6、配置IP映射
vim /etc/hosts最后增加以下內(nèi)容:
192.168.2.11 es-master
192.168.2.12 es-node1
192.168.2.13 es-node2
7、創(chuàng)建用戶
在root用戶下添加es用戶:
#添加組 groupadd es #添加用戶 adduser es -g es #設(shè)置密碼 passwd es給es用戶賦予權(quán)限:
#編輯/etc/sudoers vi /etc/sudoers #在root ALL=(ALL) ALL下增加es配置,最終如下 root ALL=(ALL) ALL es ALL=(ALL) ALL切換到es用戶,配置免密登錄:
#切換es用戶 su es #切換到es用戶根目錄 cd ~ #查看根目錄 pwd #生成rsa ssh-keygen -t rsa #生成公鑰 cat .ssh/id_rsa.pub >> .ssh/authorized_keys切換到root用戶,設(shè)置免密權(quán)限:
#文件夾權(quán)限 chmod 700 /home/es/.ssh #文件權(quán)限 chmod 600 /home/es/.ssh/authorized_keys把.ssh文件夾拷貝到其他節(jié)點(diǎn)上:
su es cd ~ scp -r .ssh/ es@es-node2:/home/es/在es-master、es-node1、es-node2上執(zhí)行ssh相互驗(yàn)證免密訪問。
二、部署es
1、部署es
在es用戶下,從https://www.elastic.co/cn/downloads/elasticsearch下載elasticsearch-7.6.1-linux-x86_64.tar.gz,上傳到服務(wù)器,并解壓縮。/data目錄為大容量磁盤,創(chuàng)建數(shù)據(jù)文件和日志文件目錄:
#創(chuàng)建數(shù)據(jù)目錄 mkdir -p /data/es/data #創(chuàng)建日志目錄 mkdir -p /data/es/logs #修改目錄所有者 chown es:es -R /data/es配置es-master,編輯elasticsearch-7.6.1/config/elasticsearch.yml,修改內(nèi)容如下:
#集群名稱 cluster.name: sdc-es-cluster #節(jié)點(diǎn)名稱 node.name: es-master #數(shù)據(jù)目錄 path.data: /data/es/data #日志目錄 path.logs: /data/es/logs #本節(jié)點(diǎn)ip network.host: 0.0.0.0 #端口 http.port: 9200 #集群主節(jié)點(diǎn)候選列表 discovery.seed_hosts: ["es-master"] #集群初始主結(jié)點(diǎn)列表 cluster.initial_master_nodes: ["es-master"] #集群?jiǎn)?dòng)到2個(gè)節(jié)點(diǎn)之前,阻止數(shù)據(jù)恢復(fù) gateway.recover_after_nodes: 3 #跨域訪問設(shè)置 http.cors.enabled: true http.cors.allow-origin: "*"配置es-node1,編輯elasticsearch-7.6.1/config/elasticsearch.yml,修改內(nèi)容如下:
#集群名稱 cluster.name: sdc-es-cluster #節(jié)點(diǎn)名稱 node.name: es-node1 #數(shù)據(jù)目錄 path.data: /data/es/data #日志目錄 path.logs: /data/es/logs #本節(jié)點(diǎn)ip network.host: 0.0.0.0 #端口 http.port: 9200 #集群主節(jié)點(diǎn)候選列表 discovery.seed_hosts: ["es-master"] #集群初始主結(jié)點(diǎn)列表 cluster.initial_master_nodes: ["es-master"] #集群?jiǎn)?dòng)到2個(gè)節(jié)點(diǎn)之前,阻止數(shù)據(jù)恢復(fù) gateway.recover_after_nodes: 3 #跨域訪問設(shè)置 http.cors.enabled: true http.cors.allow-origin: "*" #數(shù)據(jù)結(jié)點(diǎn) node.master: false配置es-node2,編輯elasticsearch-7.6.1/config/elasticsearch.yml,修改內(nèi)容如下:
#集群名稱 cluster.name: sdc-es-cluster #節(jié)點(diǎn)名稱 node.name: es-node2 #數(shù)據(jù)目錄 path.data: /data/es/data #日志目錄 path.logs: /data/es/logs #本節(jié)點(diǎn)ip network.host: 0.0.0.0 #端口 http.port: 9200 #集群主節(jié)點(diǎn)候選列表 discovery.seed_hosts: ["es-master"] #集群初始主結(jié)點(diǎn)列表 cluster.initial_master_nodes: ["es-master"] #集群?jiǎn)?dòng)到2個(gè)節(jié)點(diǎn)之前,阻止數(shù)據(jù)恢復(fù) gateway.recover_after_nodes: 3 #跨域訪問設(shè)置 http.cors.enabled: true http.cors.allow-origin: "*" #數(shù)據(jù)結(jié)點(diǎn) node.master: false啟動(dòng)es:
[es@es-master elasticsearch-7.6.1]$ ./bin/elasticsearch ERROR: [2] bootstrap checks failed [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535] [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] ERROR: Elasticsearch did not exit normally - check the logs at /data/es/logs/sdc-es-cluster.log啟動(dòng)失敗,顯示2個(gè)錯(cuò)誤。
解決:max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535],在root用戶下,編輯/etc/security/limits.conf文件,在文件最后增加以下內(nèi)容:
* soft nofile 65536 * hard nofile 65536解決:max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144],在root用戶下,編輯/etc/sysctl.conf文件,在文件最后增加以下內(nèi)容:
vm.max_map_count=262144命令行執(zhí)行命令:sysctl -p,讓配置生效。
切換到es用戶,啟動(dòng)es:
[root@es-master elasticsearch-7.6.1]# su es [es@es-master elasticsearch-7.6.1]$ ./bin/elasticsearch啟動(dòng)成功,在瀏覽器輸入:http://192.168.2.11:9200,返回如下內(nèi)容:
{"name" : "es-master","cluster_name" : "sdc-es-cluster","cluster_uuid" : "kzGmp801R_CSLHdo9j0pHQ","version" : {"number" : "7.6.1","build_flavor" : "default","build_type" : "tar","build_hash" : "aa751e09be0a5072e8570670309b1f12348f023b","build_date" : "2020-02-29T00:15:25.529771Z","build_snapshot" : false,"lucene_version" : "8.4.0","minimum_wire_compatibility_version" : "6.8.0","minimum_index_compatibility_version" : "6.0.0-beta1"},"tagline" : "You Know, for Search" }通過配置es-node1,切換到es用戶,啟動(dòng)es,在瀏覽器輸入:http://192.168.2.12:9200,返回如下內(nèi)容:
{"name" : "es-node1","cluster_name" : "sdc-es-cluster","cluster_uuid" : "YTOwCEiDToi2eitZaqCurA","version" : {"number" : "7.6.1","build_flavor" : "default","build_type" : "tar","build_hash" : "aa751e09be0a5072e8570670309b1f12348f023b","build_date" : "2020-02-29T00:15:25.529771Z","build_snapshot" : false,"lucene_version" : "8.4.0","minimum_wire_compatibility_version" : "6.8.0","minimum_index_compatibility_version" : "6.0.0-beta1"},"tagline" : "You Know, for Search" }通過配置es-node2,切換到es用戶,啟動(dòng)es,在瀏覽器輸入:http://192.168.2.13:9200,返回如下內(nèi)容:
{"name" : "es-node2","cluster_name" : "sdc-es-cluster","cluster_uuid" : "YTOwCEiDToi2eitZaqCurA","version" : {"number" : "7.6.1","build_flavor" : "default","build_type" : "tar","build_hash" : "aa751e09be0a5072e8570670309b1f12348f023b","build_date" : "2020-02-29T00:15:25.529771Z","build_snapshot" : false,"lucene_version" : "8.4.0","minimum_wire_compatibility_version" : "6.8.0","minimum_index_compatibility_version" : "6.0.0-beta1"},"tagline" : "You Know, for Search" }es后臺(tái)啟動(dòng):
#后臺(tái)啟動(dòng) [es@es-master elasticsearch-7.6.1]$ ./bin/elasticsearch -d #查看進(jìn)程 [es@es-master elasticsearch-7.6.1]$ jps 13441 Elasticsearch 13477 Jps2、部署elasticsearch-analysis-ik
es內(nèi)置的分詞器對(duì)中文不友好,從https://github.com/medcl/elasticsearch-analysis-ik/releases下載es中文分詞插件elasticsearch-analysis-ik.zip。
解壓縮elasticsearch-analysis-ik.zip,復(fù)制到插件到es的plugins目錄下,如:
[es@es-master elasticsearch-7.6.1]$ pwd /data/soft/elasticsearch-7.6.1 [es@es-master elasticsearch-7.6.1]$ ll plugins/analysis-ik/ 總用量 1428 -rw-r--r-- 1 es es 263965 3月 30 13:47 commons-codec-1.9.jar -rw-r--r-- 1 es es 61829 3月 30 13:47 commons-logging-1.2.jar drwxr-xr-x 2 es es 299 3月 30 13:47 config -rw-r--r-- 1 es es 54598 3月 30 13:47 elasticsearch-analysis-ik-7.6.1.jar -rw-r--r-- 1 es es 736658 3月 30 13:47 httpclient-4.5.2.jar -rw-r--r-- 1 es es 326724 3月 30 13:47 httpcore-4.4.4.jar -rw-r--r-- 1 es es 1805 3月 30 13:47 plugin-descriptor.properties -rw-r--r-- 1 es es 125 3月 30 13:47 plugin-security.policy重啟啟動(dòng)es。
分詞策略支持ik_max_word 和 ik_smart ,調(diào)用analyze接口測(cè)試, 如:
curl -XGET "http://192.168.2.11:9200/_analyze?pretty=true" -H 'Content-Type: application/json' -d' {"text":"陜西省人民醫(yī)院","tokenizer": "ik_smart" }'返回結(jié)果如下:
{"tokens" : [{"token" : "陜西省","start_offset" : 0,"end_offset" : 3,"type" : "CN_WORD","position" : 0},{"token" : "人民醫(yī)院","start_offset" : 3,"end_offset" : 7,"type" : "CN_WORD","position" : 1}] }ik_max_word:會(huì)將文本做最細(xì)粒度的拆分,比如會(huì)將“陜西省人民醫(yī)院”拆分為“陜西省,陜西,省人民醫(yī)院,人民醫(yī)院,人民,民醫(yī)院,醫(yī)院”,會(huì)窮盡各種可能的組合,適合 Term Query;
ik_smart: 會(huì)做最粗粒度的拆分,比如會(huì)將“陜西省人民醫(yī)院”拆分為“陜西省,人民醫(yī)院”,適合 Phrase 查詢。
3、集群可視化訪問
從http://nodejs.cn/download/下載node-v12.16.1-linux-x64.tar.xz,上傳es-master并解壓縮。配置nodejs環(huán)境變量:
#配置環(huán)境變量 vim /etc/profile #增加以下內(nèi)容 #nodejs export NODE_HOME=/data/soft/nodejs export PATH=$PATH:$NODE_HOME/bin #讓配置生效 source /etc/profile #查看node版本 node -v v12.16.1 #查看npm版本 npm -v 6.13.4從https://github.com/mobz/elasticsearch-head下載elasticsearch-head-master.zip,上傳es-master并解壓縮,在elasticsearch-head根目錄下命令行執(zhí)行:npm install下載相關(guān)依賴。
#編輯./_site/app.js vim ./_site/app.js #替換http://localhost:9200為http://192.168.2.11:9200在root用戶下,命令行執(zhí)行:npm run start(后臺(tái)啟動(dòng)命令:nohup npm run start &),啟動(dòng)elasticsearch-head。在瀏覽器輸入:http://192.168.2.11:9100,如下所示:
三、簡(jiǎn)單使用
es空間幾何字段包括geo_point和geo_shape。空間幾何坐標(biāo)最好采用WGS-84坐標(biāo)系,即經(jīng)度[-180,180],緯度:[-90,90],涉及到空間索引。
1、創(chuàng)建索引點(diǎn)
1.1、創(chuàng)建索引
curl -XPUT http://192.168.2.11:9200/poi參數(shù)如下:
{"settings": {"index": {"number_of_shards": 2,"number_of_replicas": 1}},"mappings": {"properties": {"name": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},"type": {"type": "keyword"},"geom": {"type": "geo_point"}}} }1.2、插入數(shù)據(jù)
curl -XPOST http://192.168.2.11:9200/poi/_doc/1
參數(shù)如下:
{"name": "頤和園","type": "公園","geom": {"lon": 116.272362,"lat": 39.99299} }curl -XPOST http://192.168.2.11:9200/poi/_doc/2
參數(shù)如下:
{"name": "陜西省人民醫(yī)院","type": "醫(yī)院","geom": [108.930479,34.239695] }1.3、查詢
空間查詢包括:box查詢、distance查詢、polygon查詢:
box查詢:
"filter": {"geo_bounding_box": {"geom": {"top": 45,"left": 100,"bottom": 30,"right": 120}} }distance查詢:
"filter": {"geo_distance": {"distance": "5km","geom": [108.947028,34.259424]} }polygon查詢:
"filter": {"geo_polygon": {"geom": {"points": [[100,30],[120,30],[120,45],[100,45]]}} }curl -XPOST http://192.168.2.11:9200/poi/_search
參數(shù)如下:
{"query": {"bool": {"must": {"match": {"name": "人民醫(yī)院"}},"filter": {"geo_polygon": {"geom": {"points": [[100,30],[120,30],[120,45],[100,45]]}}}}} }2、創(chuàng)建索引面
2.1、創(chuàng)建索引
curl -XPUT http://192.168.2.11:9200/image參數(shù)如下:
{"settings": {"index": {"number_of_shards": 2,"number_of_replicas": 1}},"mappings": {"properties": {"name": {"type": "text"},"geom": {"type": "geo_shape"}}} }2.2、插入數(shù)據(jù)
支持常用的wkt和geojson格式插入空間幾何圖形,要求點(diǎn)個(gè)數(shù)大于3,且首尾相同:
geojson:
{"name": "GF1_PMS2","geom": {"type" : "polygon","coordinates" : [[[100.0, 30.0], [101.0, 30.0], [101.0, 31.0], [100.0, 31.0], [100.0, 30.0]] ]} }gwt:
{"name": "GF4_PMS","geom": "POLYGON ((116.0 39.0, 117.0 39.0, 117.0 40.0, 116.0 40.0, 116.0 39.0))" }2.3、查詢
常用空間查詢,空間圖形由geojson、wkt構(gòu)建,空間關(guān)系支持:intersects(默認(rèn))、disjoint、within、contains。
envelope查詢:
"filter": {"geo_shape": {"geom": {"shape": {"type": "envelope","coordinates": [[100.0,50.0],[120.0,30.0]]},"relation": "within"}} }geojson查詢:
"filter": {"geo_shape": {"geom": {"shape": {"type": "polygon","coordinates": [[[100.0,30.0], [120.0,30.0], [120.0, 50.0], [100.0, 50.0], [100.0,30.0]]]},"relation": "intersects"}} }wkt查詢:
"filter": {"geo_shape": {"geom": {"shape":"POLYGON ((100.0 30.0, 120.0 30.0, 120.0 50.0, 100.0 50.0, 100.0 30.0))"}} }總結(jié)
以上是生活随笔為你收集整理的CentOS部署ElasticSearch7.6.1集群的全部?jī)?nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: Java实现多条相同数据合并为一条数据
- 下一篇: 达梦数据库一些疑难杂症的解决