elasticsearch和mysql的對比

https://blog.csdn.net/qq_21383435/article/details/79323383

Mapping ~ Schema

master: 負責在集群范圍內創建/刪除索引,將分片分配給這些節點. data: 用來保存數據和倒排索引,node.data=ture client: 將節點配置為客戶端節點，并充當負載平衡器，將傳入的請求路由到集群中的不同節點。node.master和node.data設置為false

https://blog.csdn.net/sdksdk0/article/details/78469190

Index templates

索引可使用預定義的模板進行創建,這個模板稱作Index templates。模板設置包括settings和mappings，通過模式匹配的方式使得多個索引重用一個模板，例如：

curl -XPUT localhost:9200/_template/template_1 -d ' {"template" : "te*","settings" : {"number_of_shards" : 1},"mappings" : {"type1" : {"_source" : {"enabled" : false }}} }

elk設計架構

參考
k8sgithub上 https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch

Elasticsearch最佳實踐建議將這些節點分成三個角色：Master 節點 - 僅用于集群管理，沒有數據，沒有HTTP API Client 節點 - 用于客戶端使用，無數據， 使用HTTP API Data 節點 - 旨在存儲和索引數據， 沒有HTTP API

分為 管理區 存儲區 讀寫區

定制鏡像

- es 0. 我把鏡像推到了dockerhub: lanny/quay.io_pires_docker-elasticsearch-kubernetes:5.6.0 1. 鏡像里用到的yaml: https://github.com/lannyMa/docker-elasticsearch-kubernetes 2. 可以安裝自定義插件后重新build- kibana 0. 用到這個人的這個鏡像 cfontes/kibana-xpack-less:5.5.0 https://hub.docker.com/r/cfontes/kibana-xpack-less/這個人的git(包含了elk的k8s yaml): https://github.com/cfontes/kubernetes-elasticsearch-cluster- k8s的elk集群的yaml https://github.com/pires/kubernetes-elasticsearch-cluster 鏡像到這個地址: https://github.com/lannyMa/kubernetes-elasticsearch-cluster內含elk 普羅 harbor的yaml,比較實用 https://github.com/cnych/k8s-repo

集群yaml里搜HTTP_ENABLE,統一打開head訪問權限

es-master.yaml

- name: HTTP_ENABLEvalue: "true"- name: HTTP_CORS_ALLOW_ORIGINvalue: "*"

創建集群

創建pv

$ cat es-data_claim.yaml kind: PersistentVolumeClaim apiVersion: v1 metadata:name: opspvcnamespace: kube-opsannotations:volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec:accessModes:- ReadWriteManyresources:requests:storage: 1Gi

先創建svc類型

后依次創建 data master client kibana

訪問es api

訪問kibana

打開kibana,插入示例數據

PUT _bulk {"index":{"_index":"lagou","_type":"job","_id":"1"}} {"title":"python分布式爬蟲開發","salary_min":15000,"city":"深圳","company":{"name":"騰訊","company_addr":"深圳市軟件園"},"publish_date":"2017-11-11","comments":15} {"index":{"_index":"lagou","_type":"job","_id":"2"}} {"title":"django開發","salary_min":15000,"city":"上海","company":{"name":"阿里","company_addr":"廣州市軟件園"},"publish_date":"2017-11-12","comments":20}

打開head查看數據

docker run -d -v /etc/localtime:/etc/localtime --restart=always -p 9100:9100 mobz/elasticsearch-head:5

查看數據

todo:Curator(館長) 管理索引的工具

Curator的主要用途:

• 獲取索引列表

• 過濾索引,刪除/快照

• 可以對索引做這些操作

name用途

Alias	別號
Allocation	分配
Close	關
Cluster Routing	群集路由
Create Index	創建索引
Delete Indices	刪除索引
Delete Snapshots	刪除快照
forceMerge	forceMerge
Index Settings	索引設置
Open	打開
Reindex	重新編制
Replicas	副本
Restore	恢復
Rollover	滾下
Shrink	收縮
Snapshot	快照

elk整體鏈條架構

es版本兼容

"minimum_wire_compatibility_version" : "5.6.0","minimum_index_compatibility_version" : "5.0.0" curl http://10.100.32.137:9200{"name" : "es-data-5c5969967-wb2b8","cluster_name" : "myesdb","cluster_uuid" : "qSps-b9dRI2ngGHBguJ44Q","version" : {"number" : "6.3.0","build_flavor" : "default","build_type" : "tar","build_hash" : "424e937","build_date" : "2018-06-11T23:38:03.357887Z","build_snapshot" : false,"lucene_version" : "7.3.1","minimum_wire_compatibility_version" : "5.6.0","minimum_index_compatibility_version" : "5.0.0"},"tagline" : "You Know, for Search" }

轉載于:https://www.cnblogs.com/iiiiher/p/8266930.html

總結

以上是生活随笔為你收集整理的[k8s]elk架构设计-k8s集群里搭建的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。