引言： 前面轉載過一篇團隊兄弟【伊利丹】寫的NN HA實驗記錄，我也基于他的環境實驗了NN HA對于Client的透明性。 本篇文章記錄的是親自配置NN HA的具體全過程，以及全面測試HA對client訪問透明性的全過程。希望對大家有幫助。

---

實驗環境： Hadoop2.2.0的4節點集群。ZK節點3個（ZK節點數最好為奇數個），hosts文件和各節點角色分配例如以下：
hosts： 192.168.66.91 master 192.168.66.92 slave1 192.168.66.93 slave2 192.168.66.94 slave3
角色分配：

?	Active NN	Standby NN	DN	JournalNode	Zookeeper	FailoverController
master	V	?	?	V	V	V
slave1	?	V	V	V	V	V
slave2	?	?	V	V	V	?
slave3	?	?	V	?	?	?

---

---

實驗過程：

---

---

1.下載穩定版Zookeeper http://mirrors.cnnic.cn/apache/zookeeper/stable/ 并解壓到hadoop集群某文件夾下，我放在了/home/yarn/下。

---

2.改動配置文件 配置文件在conf目錄中，將zoo_sample.cfg改名為zoo.cfg。并對其做響應改動。下面是改動過后的zoo.cfg # The number of milliseconds of each tick?ZK之間，或者Client和ZK之間心跳的時間間隔 tickTime=2000
# The number of ticks that the initial? # synchronization phase can take initLimit=10 # The number of ticks that can pass between? # sending a request and getting an acknowledgement syncLimit=5
# the directory where the snapshot is stored.? # do not use /tmp for storage, /tmp here is just? # example sakes. ?保存ZK數據的文件夾，請自行創建后在此處指定 dataDir=/home/yarn/Zookeeper/zoodata
# the port at which the clients will connect ?客戶端連接ZKserver的端口 clientPort=2181
# the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the? # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1
#保存ZK日志的文件夾，請自行創建后在此處指定 dataLogDir=/home/yarn/Zookeeper/zoolog
#******下面部分均為分布式ZK集群使用****** #ZK集群初始化時。Follower節點須要和Leader節點通信，initLimit配置的是Leader最多等待多少個心跳 initLimit=5?
#Leader和Follower之間發送消息、請求和應答時。最多等待多少個心跳 syncLimit=2?
#server.A=B:C:D #A是一個數字，表示這是第幾號server #B是當前server的ID或者主機名 #C是Followerserver與Leaderserver交換信息的port #D是當Leader掛掉時，又一次選舉Leader所使用的port server.1=192.168.66.91:2888:3888? server.2=192.168.66.92:2888:3888 server.3=192.168.66.93:2888:3888 #千萬注意：接下來須要在各個幾點的dataDir文件夾下建立myid文件，內容就是對應的A，也就是說，各個ZK節點的myid文件內容不同 !!!

---

3.改動各個節點的環境變量 在/etc/profile文件加入： export ZOOKEEPER_HOME=/home/yarn/Zookeeper/zookeeper-3.4.6 并為PATH加上 $ZOOKEEPER_HOME/bin 注意：export ZOOKEEPER_HOME要在PATH的上方。

---

以下開始改動Hadoop的配置文件：

---

4.改動core-site.xml <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://myhadoop</value> <description>注意：myhadoop為集群的邏輯名，需與hdfs-site.xml中的dfs.nameservices一致！</description> </property> <property> <name>hadoop.tmp.dir</name> <value>/home/yarn/Hadoop/hdfs2.0/tmp</value> </property> <property> <name>ha.zookeeper.quorum</name> <value>master:2181,slave1:2181,slave2:2181</value> ??<description>各個ZK節點的IP/host，及客戶端連接ZK的port，該port需與zoo.cfg中的clientPort一致。</description> </property> </configuration>

---

5.改動hdfs-site.xml <?

xml version="1.0" encoding="UTF-8"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. -->
<!-- Put site-specific property overrides in this file. -->
<configuration> <property> <name>dfs.nameservices</name> ? <value>myhadoop</value> <description> Comma-separated list of nameservices. as same as fs.defaultFS in core-site.xml. </description> </property>
<property> <name>dfs.ha.namenodes.myhadoop</name> <value>nn1,nn2</value> <description> The prefix for a given nameservice, contains a comma-separated list of namenodes for a given nameservice (eg EXAMPLENAMESERVICE). </description> </property>
<property> <name>dfs.namenode.rpc-address.myhadoop.nn1</name> <value>master:8020</value> <description> RPC address for nomenode1 of hadoop-test </description> </property>
<property> <name>dfs.namenode.rpc-address.myhadoop.nn2</name> <value>slave1:8020</value> <description> RPC address for nomenode2 of hadoop-test </description> </property>
<property> <name>dfs.namenode.http-address.myhadoop.nn1</name> <value>master:50070</value> <description> The address and the base port where the dfs namenode1 web ui will listen on. </description> </property>
<property> <name>dfs.namenode.http-address.myhadoop.nn2</name> <value>slave1:50070</value> <description> The address and the base port where the dfs namenode2 web ui will listen on. </description> </property>

<property> ? ? <name>dfs.namenode.servicerpc-address.myhadoop.n1</name> ? ? <value>master:53310</value> ? </property> ? <property> ? ? <name>dfs.namenode.servicerpc-address.myhadoop.n2</name> ? ? <value>slave1:53310</value> ? </property>


<property> <name>dfs.namenode.name.dir</name> <value>file:///home/yarn/Hadoop/hdfs2.0/name</value> <description>Determines where on the local filesystem the DFS name node should store the name table(fsimage). ?If this is a comma-delimited list of directories then the name table is replicated in all of the directories, for redundancy. </description> </property>
<property> <name>dfs.namenode.shared.edits.dir</name> <value>qjournal://slave1:8485;slave2:8485;slave3:8485/hadoop-journal</value> <description>A directory on shared storage between the multiple namenodes in an HA cluster. This directory will be written by the active and read by the standby in order to keep the namespaces synchronized. This directory does not need to be listed in dfs.namenode.edits.dir above. It should be left empty in a non-HA cluster. </description> </property>
<property> <name>dfs.datanode.data.dir</name> <value>file:///home/yarn/Hadoop/hdfs2.0/data</value> <description>Determines where on the local filesystem an DFS data node should store its blocks. ?If this is a comma-delimited list of directories, then data will be stored in all named directories, typically on different devices. Directories that do not exist are ignored. </description> </property>
<property> ? <name>dfs.ha.automatic-failover.enabled</name> ? <value>true</value> ? <description> ? ? Whether automatic failover is enabled. See the HDFS High ? ? Availability documentation for details on automatic HA ? ? configuration. ? </description> </property>
<property> <name>dfs.journalnode.edits.dir</name> <value>/home/yarn/Hadoop/hdfs2.0/journal/</value> </property>
<property> ? ? <name>dfs.client.failover.proxy.provider.myhadoop</name> ? ? ? ? ? ? ? ? ? ? ? ? ? <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> ? <description>Configure the name of the Java class which will be used by the DFS Client to determine which NameNode is the current Active, and therefore which NameNode is currently serving client requests.? 這個類是Client的訪問代理。是HA特性對于Client透明的關鍵！

</description> ? </property> ? ? ? ?? <property> ? ? ? ? <name>dfs.ha.fencing.methods</name> ? ? ? ? <value>sshfence</value> ? ? <description>how to communicate in the switch process</description> </property> ? ? ?? <property> ? ? ? ? <name>dfs.ha.fencing.ssh.private-key-files</name> ? ? ? ? <value>/home/yarn/.ssh/id_rsa</value> ? <description>the location stored ssh key</description> </property> ? ?? <property> ? ? <name>dfs.ha.fencing.ssh.connect-timeout</name> ? ? <value>1000</value> ? </property> ? ?? <property> ? ? <name>dfs.namenode.handler.count</name> ? ? <value>8</value> ? </property>?
</configuration>

---

6.將改動好的core-site.xml和hdfs-site.xml復制到各個hadoop節點。

---

7.啟動 （1）啟動ZK 在全部的ZK節點運行命令： zkServer.sh start
查看各個ZK的從屬關系： yarn@master:~$?zkServer.sh status JMX enabled by default Using config: /home/yarn/Zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg Mode:?follower
yarn@slave1:~$?zkServer.sh status JMX enabled by default Using config: /home/yarn/Zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg Mode:?follower
yarn@slave2:~$?zkServer.sh status JMX enabled by default Using config: /home/yarn/Zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg Mode:?leader
注意： 哪個ZK節點會成為leader是隨機的，第一次實驗時slave2成為了leader，第二次實驗時slave1成為了leader！
此時，在各個節點都能夠查看到ZK進程： yarn@master:~$ jps 3084?QuorumPeerMain 3212 Jps
（2）格式化ZK（僅第一次須要做） 隨意ZK節點上運行： hdfs zkfc -formatZK
（3）啟動ZKFC ZookeeperFailoverController是用來監控NN狀態。協助實現主備NN切換的，所以只在主備NN節點上啟動即可： hadoop-daemon.sh start zkfc
啟動后我們能夠看到ZKFC進程： yarn@master:~$ jps 3084 QuorumPeerMain 3292 Jps 3247?DFSZKFailoverController
（4）啟動用于主備NN之間同步元數據信息的共享存儲系統JournalNode 參見角色分配表，在各個JN節點上啟動： hadoop-daemon.sh start?journalnode
啟動后在各個JN節點都能夠看到JournalNode進程： yarn@master:~$ jps 3084 QuorumPeerMain 3358 Jps 3325?JournalNode 3247 DFSZKFailoverController
（5）格式化并啟動主NN 格式化： hdfs namenode -format
注意：僅僅有第一次啟動系統時需格式化。請勿反復格式化！
在主NN節點運行命令啟動NN: hadoop-daemon.sh start namenode
啟動后能夠看到NN進程： yarn@master:~$ jps 3084 QuorumPeerMain 3480 Jps 3325 JournalNode 3411?NameNode 3247 DFSZKFailoverController

（6）在備NN上同步主NN的元數據信息 hdfs namenode -bootstrapStandby
下面是正常運行時的最后部分日志： Re-format filesystem in Storage Directory /home/yarn/Hadoop/hdfs2.0/name ? (Y or N) Y 14/06/15 10:09:08 INFO common.Storage: Storage directory /home/yarn/Hadoop/hdfs2.0/name has been successfully formatted. 14/06/15 10:09:09 INFO namenode.TransferFsImage: Opening connection to http://master:50070/getimage?getimage=1&txid=935&storageInfo=-47:564636372:0:CID-d899b10e-10c9-4851-b60d-3e158e322a62 14/06/15 10:09:09 INFO namenode.TransferFsImage: Transfer took 0.11s at 63.64 KB/s 14/06/15 10:09:09 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000935 size 7545 bytes. 14/06/15 10:09:09 INFO util.ExitUtil: Exiting with status 0 14/06/15 10:09:09 INFO namenode.NameNode: SHUTDOWN_MSG:? /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at slave1/192.168.66.92 ************************************************************/
（7）啟動備NN 在備NN上運行命令： hadoop-daemon.sh start namenode
（8）設置主NN（這一步能夠省略，這是在設置手動切換NN時的步驟。ZK已經自己主動選擇一個節點作為主NN了） 到眼下為止，事實上HDFS還不知道誰是主NN，能夠通過監控頁面查看，兩個節點的NN都是Standby狀態。 以下我們須要在主NN節點上運行命令激活主NN： hdfs haadmin -transitionToActive nn1
（9）在主NN上啟動Datanode 在[nn1]上，啟動全部datanode hadoop-daemons.sh start datanode

---

8.效果驗證1--主備自己主動切換 眼下的主NN是192.168.0.91

備NN是192.168.0.92
我在主NN上kill掉NameNode進程： yarn@master:~$ jps 5161 NameNode 5085 JournalNode 5438 Jps 4987 DFSZKFailoverController 4904 QuorumPeerMain yarn@master:~$?kill 5161 yarn@master:~$ jps 5451 Jps 5085 JournalNode 4987 DFSZKFailoverController 4904 QuorumPeerMain
此時，主NN監控頁面無法訪問：

備NN自己主動切換為主NN：

---

9.效果驗證2--HA對shell的透明性 訪問邏輯名myhadoop，運行命令查看文件夾結構，不受影響： yarn@slave3:~$ hadoop dfs -ls hdfs://myhadoop/ DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it.
Found 3 items drwxr-xr-x ? - yarn supergroup ? ? ? ? ?0 2014-03-20 00:10 hdfs://myhadoop/home drwxrwx--- ? - yarn supergroup ? ? ? ? ?0 2014-03-17 20:11 hdfs://myhadoop/tmp drwxr-xr-x ? - yarn supergroup ? ? ? ? ?0 2014-03-17 20:15 hdfs://myhadoop/workspace

---

10.效果驗證3--HA對Client程序的透明性 使用自己寫的HdfsDAO.java測試，程序中將指向HDFS的path設置為： private static final String HDFS = "hdfs://myhadoop/"; 先ping myhadoop確保沒有配置hosts。然后執行程序，一切正常： yarn@master:~$ ping myhadoop ping: unknown host myhadoop yarn@master:~$ hadoop jar Desktop/hatest.jar HdfsDAO? ls: / ========================================================== name: hdfs://myhadoop/home, folder: true, size: 0 name: hdfs://myhadoop/tmp, folder: true, size: 0 name: hdfs://myhadoop/workspace, folder: true, size: 0 ==========================================================

轉載于:https://www.cnblogs.com/lytwajue/p/6814697.html

總結

以上是生活随笔為你收集整理的【甘道夫】Hadoop2.2.0 NN HA具体配置+Client透明性试验【完整版】的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。