Burrow 服務的安裝部署

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　作者：尹正杰

版權聲明：原創作品，謝絕轉載！否則將追究法律責任。

?

　　最近協助開發的同時幫忙把10個topic的數據使用5個topic的來工作。結果發現數據flume在手機數據時存在延遲消費的問題。因此遇到了不少坑，要是有個專門監控kafka consumer lag的開源軟件就好了！當然我們通過kafka-consumer-groups.sh這個命令也可以查看到相應的參數，但每次都需要敲擊命令行。無意間聽網友說有LinkedIn公司開源有一款叫做Burrow都軟件，專門是對kafka消費這延遲檢查。

?

一.Burrow概述

1>.什么是Burrow

　　Burrow是Apache Kafka的監控伴侶，它將消費者滯后檢查作為服務提供，無需指定閾值。它監控所有消費者的承諾抵消額，并根據需要計算這些消費者的狀態。提供HTTP端點以按需請求狀態，以及提供其他Kafka群集信息。還有可配置的通知程序可以通過電子郵件或HTTP調用將狀態發送到另一個服務。

2>.Burrow特征?

沒有門檻！通過滑動窗口評估組。
多個Kafka群集支持
使用Kafka提交的偏移自動監視所有使用者
可配置支持Zookeeper提交的偏移量
可配置支持Storm-committed offsets
消費者組狀態的HTTP端點，以及代理和消費者信息
可配置的電子郵件程序，用于發送特定組的警報
可配置的HTTP客戶端，用于向所有組的另一個系統發送警報

3>.先決條件

　　Burrow是用Go編寫的，所以在開始之前，你應該安裝并設置Go。

??

二.安裝Golang環境

1>.下載 go 的安裝包?

[root@node101.yinzhengjie.org.cn ~/Downloads]# wget https://dl.google.com/go/go1.12.1.linux-amd64.tar.gz --2019-03-28 16:09:35-- https://dl.google.com/go/go1.12.1.linux-amd64.tar.gz Resolving dl.google.com (dl.google.com)... 203.208.41.70, 203.208.41.72, 203.208.41.65, ... Connecting to dl.google.com (dl.google.com)|203.208.41.70|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 127906702 (122M) [application/octet-stream] Saving to: ‘go1.12.1.linux-amd64.tar.gz’100%[=========================================================================================================================================================================>] 127,906,702 2.27MB/s in 54s 2019-03-28 16:10:29 (2.28 MB/s) - ‘go1.12.1.linux-amd64.tar.gz’ saved [127906702/127906702][root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# wget https://dl.google.com/go/go1.12.1.linux-amd64.tar.gz

2>.解壓go的安裝包到指定目錄

[root@node101.yinzhengjie.org.cn ~/Downloads]# tar -zxf go1.12.1.linux-amd64.tar.gz -C /yinzhengjie/softwares/ [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# ll /yinzhengjie/softwares/go/ total 208 drwxr-xr-x. 2 root root 4096 Mar 15 03:43 api -rw-r--r--. 1 root root 55358 Mar 15 03:43 AUTHORS drwxr-xr-x. 2 root root 4096 Mar 15 03:46 bin -rw-r--r--. 1 root root 1339 Mar 15 03:43 CONTRIBUTING.md -rw-r--r--. 1 root root 78132 Mar 15 03:43 CONTRIBUTORS drwxr-xr-x. 8 root root 4096 Mar 15 03:43 doc -rw-r--r--. 1 root root 5686 Mar 15 03:43 favicon.ico drwxr-xr-x. 3 root root 4096 Mar 15 03:43 lib -rw-r--r--. 1 root root 1479 Mar 15 03:43 LICENSE drwxr-xr-x. 13 root root 4096 Mar 15 03:43 misc -rw-r--r--. 1 root root 1303 Mar 15 03:43 PATENTS drwxr-xr-x. 6 root root 4096 Mar 15 03:46 pkg -rw-r--r--. 1 root root 1607 Mar 15 03:43 README.md -rw-r--r--. 1 root root 26 Mar 15 03:43 robots.txt drwxr-xr-x. 46 root root 4096 Mar 15 03:43 src drwxr-xr-x. 21 root root 12288 Mar 15 03:43 test -rw-r--r--. 1 root root 8 Mar 15 03:43 VERSION [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# tar -zxf go1.12.1.linux-amd64.tar.gz -C /yinzhengjie/softwares/

3>.配置go到環境變量?

[root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# mkdir /yinzhengjie/goproject [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# tail -5 /etc/profile #ADD GO_PATH export GO_HOME=/yinzhengjie/softwares/go export PATH=$PATH:$GO_HOME/bin export GOPATH=/yinzhengjie/goproject[root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# tail -5 /etc/profile [root@node101.yinzhengjie.org.cn ~/Downloads]# source /etc/profile [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# go env GOARCH="amd64" GOBIN="" GOCACHE="/root/.cache/go-build" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOOS="linux" GOPATH="/yinzhengjie/goproject" GOPROXY="" GORACE="" GOROOT="/yinzhengjie/softwares/go" GOTMPDIR="" GOTOOLDIR="/yinzhengjie/softwares/go/pkg/tool/linux_amd64" GCCGO="gccgo" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build685793832=/tmp/go-build -gno-record-gcc-switches" [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# go env

?

三.Burrow安裝

1>.Burrow的github地址（https://github.com/linkedin/Burrow）

2>.安裝git工具

[root@node101.yinzhengjie.org.cn ~]# yum -y install git Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile* base: mirrors.aliyun.com* extras: mirrors.aliyun.com* updates: mirrors.aliyun.com Resolving Dependencies --> Running transaction check ---> Package git.x86_64 0:1.8.3.1-20.el7 will be installed --> Processing Dependency: perl-Git = 1.8.3.1-20.el7 for package: git-1.8.3.1-20.el7.x86_64 --> Processing Dependency: perl(Term::ReadKey) for package: git-1.8.3.1-20.el7.x86_64 --> Processing Dependency: perl(Git) for package: git-1.8.3.1-20.el7.x86_64 --> Processing Dependency: perl(Error) for package: git-1.8.3.1-20.el7.x86_64 --> Running transaction check ---> Package perl-Error.noarch 1:0.17020-2.el7 will be installed ---> Package perl-Git.noarch 0:1.8.3.1-20.el7 will be installed ---> Package perl-TermReadKey.x86_64 0:2.30-20.el7 will be installed --> Finished Dependency ResolutionDependencies Resolved===================================================================================================================================================================================================================Package Arch Version Repository Size =================================================================================================================================================================================================================== Installing:git x86_64 1.8.3.1-20.el7 updates 4.4 M Installing for dependencies:perl-Error noarch 1:0.17020-2.el7 base 32 kperl-Git noarch 1.8.3.1-20.el7 updates 55 kperl-TermReadKey x86_64 2.30-20.el7 base 31 kTransaction Summary =================================================================================================================================================================================================================== Install 1 Package (+3 Dependent packages)Total download size: 4.5 M Installed size: 22 M Downloading packages: (1/4): perl-Error-0.17020-2.el7.noarch.rpm | 32 kB 00:00:00 perl-TermReadKey-2.30-20.el7.x FAILED http://mirrors.cloud.aliyuncs.com/centos/7/os/x86_64/Packages/perl-TermReadKey-2.30-20.el7.x86_64.rpm: [Errno 14] curl#6 - "Could not resolve host: mirrors.cloud.aliyuncs.com; Unknown error" 0 B --:--:-- ETA Trying other mirror. (2/4): perl-TermReadKey-2.30-20.el7.x86_64.rpm | 31 kB 00:00:00 perl-Git-1.8.3.1-20.el7.noarch FAILED http://mirrors.cloud.aliyuncs.com/centos/7/updates/x86_64/Packages/perl-Git-1.8.3.1-20.el7.noarch.rpm: [Errno 14] curl#6 - "Could not resolve host: mirrors.cloud.aliyuncs.com; Unknown error" 0 B --:--:-- ETA Trying other mirror. (3/4): git-1.8.3.1-20.el7.x86_64.rpm | 4.4 MB 00:00:01 (4/4): perl-Git-1.8.3.1-20.el7.noarch.rpm | 55 kB 00:00:01 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 2.1 MB/s | 4.5 MB 00:00:02 Running transaction check Running transaction test Transaction test succeeded Running transactionInstalling : 1:perl-Error-0.17020-2.el7.noarch 1/4 Installing : perl-TermReadKey-2.30-20.el7.x86_64 2/4 Installing : git-1.8.3.1-20.el7.x86_64 3/4 Installing : perl-Git-1.8.3.1-20.el7.noarch 4/4 Verifying : perl-Git-1.8.3.1-20.el7.noarch 1/4 Verifying : 1:perl-Error-0.17020-2.el7.noarch 2/4 Verifying : git-1.8.3.1-20.el7.x86_64 3/4 Verifying : perl-TermReadKey-2.30-20.el7.x86_64 4/4 Installed:git.x86_64 0:1.8.3.1-20.el7 Dependency Installed:perl-Error.noarch 1:0.17020-2.el7 perl-Git.noarch 0:1.8.3.1-20.el7 perl-TermReadKey.x86_64 0:2.30-20.el7 Complete! [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# yum -y install git

2>.

?

?

3>.

?

4>.

?

5>.?

轉載于:https://www.cnblogs.com/yinzhengjie/p/10544629.html

總結

以上是生活随笔為你收集整理的Burrow 服务的安装部署的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。