搭建HDFS

增加主機名

我這里僅僅增加了master主機名

[root@10 /xinghl/hadoop/bin]$ cat /etc/hosts 127.0.0.1 localhost 10.0.67.101 ::1 localhost 10.0.67.101 10.0.67.101 master 10.0.67.101

如果配置遠程集群，則需要配置SSH

我這里就是單節(jié)點測試玩玩，所以這步驟先省略了。

解壓縮hadoop到/usr目錄下

創(chuàng)建幾個需要的目錄

mkdir /dfs mkdir /dfs/name mkdir /dfs/data mkdir /tmp

修改配置文件，在$HADOOP_HOME/etc/hadoop下

修改hadoop-env.sh

export JAVA_HOME=/usr/java

修改slaves

我這里就是

localhost

修改core-site.xml

<configuration><property><name>fs.defaultFS</name><value>hdfs://master:8020</value></property><property><name>io.file.buffer.size</name><value>131072</value></property><property><name>hadoop.tmp.dir</name><value>file:/usr/hadoop/tmp</value><description>Abase for other temporary directories.</description></property><property><name>hadoop.proxyuser.u0.hosts</name><value>*</value></property><property><name>hadoop.proxyuser.u0.groups</name><value>*</value></property> </configuration>

修改hdfs-site.xml

<configuration><property><name>dfs.namenode.secondary.http-address</name><value>master:9001</value></property><property><name>dfs.namenode.name.dir</name><value>file:/usr/hadoop/dfs/name</value></property><property><name>dfs.datanode.data.dir</name><value>file:/usr/hadoop/dfs/data</value></property><property><name>dfs.replication</name><value>3</value></property><property><name>dfs.webhdfs.enabled</name><value>true</value></property> </configuration>

啟動hadoop

hadoop namenode -format #在$HADOOP_HOME/bin目錄下 start-all.sh #在$HADOOP_HOME/sbin目錄下

檢查運行狀態(tài)

1 使用jps命令查看
2 登錄http://10.0.67.101:8088/cluster

配置SSH，支持無密碼登錄

[root@localhost sbin]# cd ~/.ssh/ [root@localhost .ssh]# ll total 0 [root@localhost .ssh]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 75:ea:4f:ec:df:55:06:3c:af:91:12:50:69:a5:86:27 root@localhost.localdomain The key's randomart image is: +--[ RSA 2048]----+ | ...o. | | ooo | | E.* + | | . * . = | | S . . o +| | . . . +.| | . o . .| | + ..| | o.. .| +-----------------+ [root@localhost .ssh]# [root@localhost .ssh]# ll total 8 -rw-------. 1 root root 1675 Aug 16 10:55 id_rsa -rw-r--r--. 1 root root 408 Aug 16 10:55 id_rsa.pub [root@localhost .ssh]# cat id_rsa.pub >> ~/.ssh/authorized_keys [root@localhost .ssh]# ll total 12 -rw-r--r--. 1 root root 408 Aug 16 10:55 authorized_keys -rw-------. 1 root root 1675 Aug 16 10:55 id_rsa -rw-r--r--. 1 root root 408 Aug 16 10:55 id_rsa.pub [root@localhost .ssh]# ssh localhost The authenticity of host 'localhost (::1)' can't be established. RSA key fingerprint is 9e:7b:cb:c5:06:e9:81:e5:db:57:9c:f9:79:a2:c4:ce. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'localhost' (RSA) to the list of known hosts. [root@localhost ~]# exit logout Connection to localhost closed. [root@localhost .ssh]# ssh localhost Last login: Tue Aug 16 10:55:30 2016 from localhost [root@localhost ~]# exit logout Connection to localhost closed. [root@localhost .ssh]#

轉(zhuǎn)載于:https://www.cnblogs.com/xing901022/p/5713585.html

總結(jié)

以上是生活随笔為你收集整理的单节点部署Hadoop教程的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。