文章目錄

• • 建立連接獲取hadoop下的文件信息
  • 編寫hadoop連接工具類

建立連接獲取hadoop下的文件信息

import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.BlockLocation; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.LocatedFileStatus; import org.apache.hadoop.fs.Path; import org.apache.hadoop.fs.RemoteIterator;public class ListFile {public static void main(String [] args) throws IOException, Exception,URISyntaxException{FileSystem file = FileSystem.get(new URI("hdfs://虛擬機(jī)IP地址:9000"),new Configuration(),"hadoop");//獲取FileSystem對(duì)象RemoteIterator<LocatedFileStatus> iterator = file.listFiles(new Path("/"), true);//調(diào)用listFiles 獲取 /目錄下所有的文件信息while(iterator.hasNext()) {//遍歷 / 文件夾LocatedFileStatus fileStatus = iterator.next();//獲取文件狀態(tài)對(duì)象Path path = fileStatus.getPath();//獲取決定路徑String name = path.getName();//獲取名字System.out.println(path.toString()+" "+path.getName());BlockLocation[] locations = fileStatus.getBlockLocations();//獲取文件塊的信息System.out.println("block"+locations.length);//輸出文件塊的大小}} }

編寫hadoop連接工具類

package com.clouddisk.cloud.util;import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.*;import java.io.IOException; import java.net.URI; import java.net.URISyntaxException;public class HDFSUtils {private static Configuration conf ;private static FileSystem fs ;private static String address = "hdfs://192.168.43.60:9000";public FileSystem getConf() throws URISyntaxException, IOException, InterruptedException {conf = new Configuration();fs = FileSystem.get(new URI(address),conf,"hadoop");return fs;}public void release() throws IOException {fs.close();}public String getAddress(){return address;} }

總結(jié)

以上是生活随笔為你收集整理的HDFS学习 Java连接hadoop的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。