一、基本作用

概念: Sqoop被稱為協(xié)作框架,是在Hadoop.2.X生態(tài)系統(tǒng)的輔助型框架,簡單說,就是一個(gè)數(shù)據(jù)轉(zhuǎn)換工具,類似的協(xié)作框架有文件收集庫框架Flume,任務(wù)協(xié)調(diào)框架Oozie,大數(shù)據(jù)Web工具Hue 過程: 數(shù)據(jù)源(RDBMS)取得數(shù)據(jù)<--->數(shù)據(jù)清洗/數(shù)據(jù)分析<--->HDFS/HBASE/HDFS 作用: Sql-to-Hadoop,是連接關(guān)系型數(shù)據(jù)庫和Hadoop的橋梁,以mapreduce為底層,通過參數(shù)與與mapreduce模板封裝成jar包,提交給Yarn,利用MapReduce加快數(shù)據(jù)傳輸速度,批處理方式進(jìn)行數(shù)據(jù)傳輸 版本: 1.4.x 為Sqoop1 1.99.x為Sqoop2
二進(jìn)制下載包下載地址: http://archive.cloudera.com/cdh5/cdh/5/

二、簡單配置

sqoop-1.4.5-cdh5.3.6/conf sqoop-env.sh export HADOOP_COMMON_HOME=/opt/cdh-5.6.3/hadoop-2.5.0-cdh5.3.6 export HADOOP_MAPRED_HOME=/opt/cdh-5.6.3/hadoop-2.5.0-cdh5.3.6 export HIVE_HOME=/opt/cdh-5.6.3/hive-0.13.1-cdh5.3.6

三、簡單使用

# 連接mysql數(shù)據(jù)庫時(shí)注意將mysql的驅(qū)動(dòng)jar包放入lib目錄下 $ bin/sqoop help Available commands:codegen Generate code to interact with database recordscreate-hive-table Import a table definition into Hiveeval Evaluate a SQL statement and display the resultsexport Export an HDFS directory to a database tablehelp List available commandsimport Import a table from a database to HDFSimport-all-tables Import tables from a database to HDFSimport-mainframe Import datasets from a mainframe server to HDFSjob Work with saved jobslist-databases List available databases on a serverlist-tables List available tables in a databasemerge Merge results of incremental importsmetastore Run a standalone Sqoop metastoreversion Display version information $ bin/sqoop list-databases --connect jdbc:mysql://10.0.0.108:3306 --username root --password root $ bin/sqoop list-tables --connect jdbc:mysql://10.0.0.108:3306/mysql --username root --password root $ bin/sqoop import --help $ bin/sqoop export --help

轉(zhuǎn)載于:https://www.cnblogs.com/eRrsr/p/6051014.html

總結(jié)

以上是生活随笔為你收集整理的Sqoop_ 简单介绍的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。