Azkaban 兩個(gè)服務(wù)器模式安裝與使用

需要的工具

Azkaban Web服務(wù)安裝包 azkaban-web-server-0.1.0-SNAPSHOT.tar.gz

Azkaban執(zhí)行服務(wù)安裝包 azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz

編譯之后的sql腳本 create-all-sql-0.1.0-SNAPSHOT.sql

C程序文件腳本 execute-as-user.c程序

Step 1: 數(shù)據(jù)庫準(zhǔn)備

進(jìn)入mysql的客戶端執(zhí)行以下命令

mysql ‐uroot ‐p

執(zhí)行以下命令：

CREATE DATABASE azkaban; CREATE USER 'azkaban'@'%' IDENTIFIED BY 'azkaban'; GRANT all privileges ON azkaban.* to 'azkaban'@'%' identified by 'azkaban' WITH GRANT OPTION; flush privileges; use azkaban; source /export/software/create‐all‐sql‐0.1.0‐SNAPSHOT.sql;

Step 2: 解壓軟件包

解壓軟件安裝包

解壓azkaban-web-server

cd /export/softwares tar ‐zxvf azkaban‐web‐server‐0.1.0‐SNAPSHOT.tar.gz ‐C ../servers/ cd /export/servers mv azkaban‐web‐server‐0.1.0‐SNAPSHOT/ azkaban‐web‐server‐3.51.0 * 解壓azkaban‐exec‐server ```shell cd /export/softwares tar ‐zxvf azkaban‐exec‐server‐0.1.0‐SNAPSHOT.tar.gz ‐C ../servers/ cd /export/servers mv azkaban‐exec‐server‐0.1.0‐SNAPSHOT/ azkaban‐exec‐server‐3.51.0

Step 3: 安裝SSL安全認(rèn)證

安裝ssl安全認(rèn)證，允許我們使用https的方式訪問我們的azkaban的web服務(wù)

密碼一定要一個(gè)個(gè)的字母輸入，或者粘貼也行

cd /export/servers/azkaban-web-server-3.51.0 keytool -keystore keystore -alias jetty -genkey -keyalg RSA

Step 4: azkaban web server安裝

修改azkaban-web-server的配置文件

cd /export/servers/azkaban‐web‐server‐3.51.0/conf vim azkaban.properties # Azkaban Personalization Settings azkaban.name=Azkaban azkaban.label=My Azkaban azkaban.color=#FF3601 azkaban.default.servlet.path=/index web.resource.dir=web/ default.timezone.id=Asia/Shanghai # Azkaban UserManager class user.manager.class=azkaban.user.XmlUserManager user.manager.xml.file=conf/azkaban‐users.xml # Loader for projects executor.global.properties=conf/global.properties azkaban.project.dir=projects # Velocity dev mode velocity.dev.mode=false # Azkaban Jetty server properties. jetty.use.ssl=true jetty.maxThreads=25 jetty.port=8081jetty.ssl.port=8443 jetty.keystore=/export/servers/azkaban‐web‐server‐3.51.0/keystore jetty.password=azkaban jetty.keypassword=azkaban jetty.truststore=/export/servers/azkaban‐web‐server‐3.51.0/keystore jetty.trustpassword=azkaban # Azkaban Executor settings # mail settings mail.sender= mail.host= # User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users. # enduser ‐> myazkabanhost:443 ‐> proxy ‐> localhost:8081 # when this parameters set then these parameters are used to generate email links.# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used. # azkaban.webserver.external_hostname=myazkabanhost.com # azkaban.webserver.external_ssl_port=443 # azkaban.webserver.external_port=8081 job.failure.email= job.success.email= lockdown.create.projects=false cache.directory=cache # JMX stats jetty.connector.stats=true executor.connector.stats=true # Azkaban mysql settings by default. Users should configure their own username and password. database.type=mysql mysql.port=3306 mysql.host=node03 mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100 #Multiple Executor azkaban.use.multiple.executors=true #azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemo ry,CpuStatus azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1 azkaban.executorselector.comparator.Memory=1 azkaban.executorselector.comparator.LastDispatched=1 azkaban.executorselector.comparator.CpuUsage=1azkaban.activeexecutor.refresh.milisecinterval=10000 azkaban.queueprocessing.enabled=true azkaban.activeexecutor.refresh.flowinterval=10 azkaban.executorinfo.refresh.maxThreads=10

Step 5: azkaban executor server 安裝

第一步：修改azkaban-exex-server配置文件

修改azkaban-exec-server的配置文件

cd /export/servers/azkaban‐exec‐server‐3.51.0/conf vim azkaban.properties # Azkaban Personalization Settings azkaban.name=Azkaban azkaban.label=My Azkaban azkaban.color=#FF3601 azkaban.default.servlet.path=/index web.resource.dir=web/ default.timezone.id=Asia/Shanghai # Azkaban UserManager class user.manager.class=azkaban.user.XmlUserManager user.manager.xml.file=conf/azkaban‐users.xml # Loader for projects executor.global.properties=conf/global.properties azkaban.project.dir=projects # Velocity dev mode velocity.dev.mode=false # Azkaban Jetty server properties. jetty.use.ssl=true jetty.maxThreads=25 jetty.port=8081jetty.keystore=/export/servers/azkaban‐web‐server‐3.51.0/keystore jetty.password=azkaban jetty.keypassword=azkaban jetty.truststore=/export/servers/azkaban‐web‐server‐3.51.0/keystore jetty.trustpassword=azkaban# Where the Azkaban web server is located azkaban.webserver.url=https://node03:8443 # mail settings mail.sender= mail.host= # User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users. # enduser ‐> myazkabanhost:443 ‐> proxy ‐> localhost:8081 # when this parameters set then these parameters are used to generate email links# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used. # azkaban.webserver.external_hostname=myazkabanhost.com # azkaban.webserver.external_ssl_port=443 # azkaban.webserver.external_port=8081 job.failure.email= job.success.email= lockdown.create.projects=false cache.directory=cache # JMX stats jetty.connector.stats=true executor.connector.stats=true # Azkaban plugin settings azkaban.jobtype.plugin.dir=plugins/jobtypes # Azkaban mysql settings by default. Users should configure their own username and password. database.type=mysql mysql.port=3306 mysql.host=node03 mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100 # Azkaban Executor settings executor.maxThreads=50 executor.flow.threads=30

Step 6: azkaban executor server 安裝

第二步：添加插件

將我們編譯后的C文件execute-as-user.c上傳到這個(gè)目錄來/export/servers/azkaban-exec-server-3.51.0/plugins/jobtypes或者直接將我們/export/softwares下面的文件拷貝過來也行

cp /export/softwares/execute‐as‐user.c /export/servers/azkaban‐execserver‐ 3.51.0/plugins/jobtypes/

然后執(zhí)行以下命令生成execute-as-user

yum ‐y install gcc‐c++ cd /export/servers/azkaban‐exec‐server‐3.51.0/plugins/jobtypes gcc execute‐as‐user.c ‐o execute‐as‐user chown root execute‐as‐user chmod 6050 execute‐as‐user

Step 7: azkaban executor server 安裝

第三步：修改配置文件

修改配置文件

cd /export/servers/azkaban‐exec‐server‐3.47.0/plugins/jobtypes vim commonprivate.properties execute.as.user=false memCheck.enabled=false azkaban.native.lib=/export/servers/azkaban‐exec‐server‐ 3.51.0/plugins/jobtypes

Step 7: 啟動(dòng)服務(wù)

第一步：啟動(dòng)azkaban exec server

cd /export/servers/azkaban‐exec‐server‐3.51.0 bin/start‐exec.sh

第二步：激活我們的exec-server

node03機(jī)器任意目錄下執(zhí)行以下命令

curl ‐G "node03:$(<./executor.port)/executor?action=activate" && echo

第三步：啟動(dòng)azkaban-web-server

cd /export/servers/azkaban‐web‐server‐3.51.0/ bin/start‐web.sh

訪問地址：

https://node03:8443

Step 8: 修改linux的時(shí)區(qū)問題

由于先前做好了時(shí)鐘同步，所以不用擔(dān)心時(shí)區(qū)問題，不需要修改時(shí)區(qū)了

注：先配置好服務(wù)器節(jié)點(diǎn)上的

1. 先生成時(shí)區(qū)配置文件Asia/Shanghai，用交互式命令 tzselect 即可

2. 拷貝該時(shí)區(qū)文件，覆蓋系統(tǒng)本地時(shí)區(qū)配置

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

?

總結(jié)

以上是生活随笔為你收集整理的Azkaban-two_server模式-安装3和启动运行的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。