世界上并沒有完美的程序，但是我們并不因此而沮喪，因為寫程序就是一個不斷追求完美的過程。
自定義分析器 :
Character filters :
????1. 作用 : 字符的增、刪、改轉換
????2. 數量限制 : 可以有0個或多個
????3. 內建字符過濾器 :
????????1. HTML Strip Character filter : 去除html標簽
????????2. Mapping Character filter : 映射替換
????????3. Pattern Replace Character filter : 正則替換Tokenizer :
????1. 作用 :
????????1. 分詞
????????2. 記錄詞的順序和位置（短語查詢）
????????3. 記錄詞的開頭和結尾位置（高亮）
????????4. 記錄詞的類型（分類）
????2. 數量限制 : 有且只能有一個
????3. 分類 :
????????1. 完整分詞 :
????????????1. Standard
????????????2. Letter
????????????3. Lowercase
????????????4. whitespace
????????????5. UAX URL Email
????????????6. Classic
????????????7. Thai
????????2. 切詞 :
????????????1. N-Gram
????????????2. Edge N-Gram
????????3. 文本 :
????????????1. Keyword
????????????2. Pattern
????????????3. Simple Pattern
????????????4. Char Group
????????????5. Simple Pattern split
????????????6. PathToken filters :
????1. 作用 : 分詞的增、刪、改轉換
????2. 數量限制 : 可以有0個或多個
????3. 分類 :
????????1. apostrophe
????????2. asciifolding
????????3. cjk bigram
????????4. cjk width
????????5. classic
????????6. common grams
????????7. conditional
????????8. decimal digit
????????9. delimited payload
????????10. dictionary decompounder
????????11. edge ngram
????????12. elision
????????13. fingerprint
????????14. flatten_graph
????????15. hunspell
????????16. hyphenation decompounder
????????17. keep types
????????18. keep words
????????19. keyword marker
????????20. keyword repeat
????????21. kstem
????????22. length
????????23. limit token count
????????24. lowercase
????????25. min_hash
????????26. multiplexer
????????27. ngram
????????28. normalization
????????29. pattern_capture
????????30. pattern replace
????????31. porter stem
????????32. predicate script
????????33. remove duplicates
????????34. reverse
????????35. shingle
????????36. snowball
????????37. stemmer
????????38. stemmer override
????????39. stop
????????40. synonym
????????41. synonym graph
今天演示38 - 41
重點 : stop, synonym
# stemmer override token filter
# 作用
: 自定義詞干映射
# 條件
: 必須放在所有詞干提取器之前
# 配置項
:
#
1. rules
: 映射
#
2. rules_path
: 映射路徑GET
/_analyze
{"tokenizer" : "whitespace","filter" : [{"type" : "stemmer_override","rules" : ["gooding, goodly => good","hello => hi","中國 => 中華人民共和國"]}],"text" : ["hello gooding me 中國"]
}# 結果
{"tokens" : [{"token" : "hi","start_offset" : 0,"end_offset" : 5,"type" : "word","position" : 0},{"token" : "good","start_offset" : 6,"end_offset" : 13,"type" : "word","position" : 1},{"token" : "me","start_offset" : 14,"end_offset" : 16,"type" : "word","position" : 2},{"token" : "中華人民共和國","start_offset" : 17,"end_offset" : 19,"type" : "word","position" : 3}]
}
# stop token filter
# 作用
: 移除停用詞
# 配置項
:
#
1. stopwords
: 停用詞列表或語言
#
2. stopwords_path
: 停用詞文件路徑
#
3. ignore_case
: 是否忽略大小寫，默認
false
#
4. remove_trailing
: 移除末尾的停用詞，默認
false，但是在completion suggester中建議設置為
true以便更好的匹配GET
/_analyze
{"tokenizer" : "whitespace","filter" : [{"type" : "stop","stopwords" : ["this", "is", "a"]}],"text" : ["this is a good boy"]
}# 結果
{"tokens" : [{"token" : "good","start_offset" : 10,"end_offset" : 14,"type" : "word","position" : 3},{"token" : "boy","start_offset" : 15,"end_offset" : 18,"type" : "word","position" : 4}]
}
# synonym token filter
# 作用
: 添加同義詞
# 配置項
:
#
1. synonyms
: 同義詞列表，支持solr和wordnet格式
#
2. synonyms_path
: 同義詞文件路徑
# 使用建議
: 搜索時使用同義詞，索引時不使用同義詞GET
/_analyze
{"tokenizer" : "whitespace","filter" : [{"type" : "synonym","synonyms" : ["hello, hi => hell, he", "中國, 中國人, 我是中國人"]}],"text" : ["hello gooding me hi this is me 中國"]
}# 結果
{"tokens" : [{"token" : "hell","start_offset" : 0,"end_offset" : 5,"type" : "SYNONYM","position" : 0},{"token" : "he","start_offset" : 0,"end_offset" : 5,"type" : "SYNONYM","position" : 0},{"token" : "gooding","start_offset" : 6,"end_offset" : 13,"type" : "word","position" : 1},{"token" : "me","start_offset" : 14,"end_offset" : 16,"type" : "word","position" : 2},{"token" : "hell","start_offset" : 17,"end_offset" : 19,"type" : "SYNONYM","position" : 3},{"token" : "he","start_offset" : 17,"end_offset" : 19,"type" : "SYNONYM","position" : 3},{"token" : "this","start_offset" : 20,"end_offset" : 24,"type" : "word","position" : 4},{"token" : "is","start_offset" : 25,"end_offset" : 27,"type" : "word","position" : 5},{"token" : "me","start_offset" : 28,"end_offset" : 30,"type" : "word","position" : 6},{"token" : "中國","start_offset" : 31,"end_offset" : 33,"type" : "word","position" : 7},{"token" : "中國人","start_offset" : 31,"end_offset" : 33,"type" : "SYNONYM","position" : 7},{"token" : "我是中國人","start_offset" : 31,"end_offset" : 33,"type" : "SYNONYM","position" : 7}]
}
# synonym graph token filter
# 作用
: 添加同義詞，只支持搜索分析器
# 配置項
:
#
1. synonyms
: 同義詞列表，支持solr和wordnet格式
#
2. synonyms_path
: 同義詞文件路徑GET
/_analyze
{"tokenizer" : "whitespace","filter" : [{"type" : "synonym_graph","synonyms" : ["hello, hi => hell, he", "中國, 中國人, 我是中國人"]}],"text" : ["hello gooding me hi this is me 中國"]
}# 結果
{"tokens" : [{"token" : "hell","start_offset" : 0,"end_offset" : 5,"type" : "SYNONYM","position" : 0},{"token" : "he","start_offset" : 0,"end_offset" : 5,"type" : "SYNONYM","position" : 0},{"token" : "gooding","start_offset" : 6,"end_offset" : 13,"type" : "word","position" : 1},{"token" : "me","start_offset" : 14,"end_offset" : 16,"type" : "word","position" : 2},{"token" : "hell","start_offset" : 17,"end_offset" : 19,"type" : "SYNONYM","position" : 3},{"token" : "he","start_offset" : 17,"end_offset" : 19,"type" : "SYNONYM","position" : 3},{"token" : "this","start_offset" : 20,"end_offset" : 24,"type" : "word","position" : 4},{"token" : "is","start_offset" : 25,"end_offset" : 27,"type" : "word","position" : 5},{"token" : "me","start_offset" : 28,"end_offset" : 30,"type" : "word","position" : 6},{"token" : "中國人","start_offset" : 31,"end_offset" : 33,"type" : "SYNONYM","position" : 7},{"token" : "我是中國人","start_offset" : 31,"end_offset" : 33,"type" : "SYNONYM","position" : 7},{"token" : "中國","start_offset" : 31,"end_offset" : 33,"type" : "word","position" : 7}]
}
總結
以上是生活随笔為你收集整理的es - elasticsearch 自定义分析器 - 内建分词过滤器 - 11的全部內容，希望文章能夠幫你解決所遇到的問題。
如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。
