Hadoop中-put和-copyFromLocal的区别
如下中的stackoverflow的鏈接。
簡單的說，-put更寬松，可以把本地或者HDFS上的文件拷貝到HDFS中；而-copyFromLocal則更嚴(yán)格限制只能拷貝本地文件到HDFS中。
？？？
PS：“?put would prefer the HDFS scheme instead of the local file system”，也就是說，如果本地和HDFS上都存在相同路徑，則-put跟趨于優(yōu)先取HDFS的源。
但是我測試了：
hadoop fs -put hdfs:///tmp/hive-XXX/test.txt /user/XXX/test.txt.hdfs
hadoop fs -put /tmp/hive-XXX/test.txt /user/XXX/test.txt.local ? ? ??
hadoop fs -cat /user/XXX/test.txt.* ? ?
local path:/tmp/hive-XXX
local path:/tmp/hive-XXX
所以。。。。
鏈接：http://stackoverflow.com/questions/7811284/difference-between-hadoop-fs-put-and-hadoop-fs-copyfromlocal
——————————————————————————————————————————————
Difference between hadoop fs -put and hadoop fs -copyFromLocal
id="ados_frame_adzerk307105374_547380" frameborder="0" scrolling="no" height="90" width="728" style="margin: 0px; padding: 0px; border-width: 0px;">| up vote17down votefavorite2 | -put?and?-copyFromLocal?are documented as identical, while most examples use the verbose variant -copyFromLocal. Why? Same thing for?-get?and?-copyToLocal hadoop?hdfs
| ||
| ? | add a comment |
2 Answers
activeoldestvotes| up vote27down voteaccepted |
So, basically you can do with?put, all that you do with?copyFromLocal, but not vice-versa. Similarly,
Hence, you can use?get?instead of?copyToLocal, but not the other way round. Reference:?Hadoop's documentation.
| |||
| ? | add a comment |
| up vote20down vote | Let's make an example: If your HDFS contains the path:?/tmp/dir/abc.txt?And if your local disk also contains this path then the hdfs API won't know which one you mean, unless you specify a scheme like?file://?or?hdfs://. Maybe it picks the path you did not want to copy. Therefore you have?-copyFromLocal?which is preventing you from accidentally copying the wrong file, by limiting the parameter you give to the local filesystem. Put?is for more advanced users who know which scheme to put in front. It is always a bit confusing to new Hadoop users which filesystem they are currently in and where their files actually are.
| ||||||||||||||||||||
| ? |
|
總結(jié)
以上是生活随笔為你收集整理的Hadoop中-put和-copyFromLocal的区别的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： Hadoop Stream Python
- 下一篇： 蹬N级阶梯有几种走法