轉自：https://www.iteye.com/blog/vase-2090320
不知道是不是hive-0.12版增強了local mode的原因，在之前版本運行好好的Hive-QL在這個版本上錯誤頻頻，折磨一天多以后終于定位到原因，把在內部的總結在這再記錄下，希望對遇到同樣問題的筒子們有所幫助。

部分一 關于return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask

Hive升級到0.12版之后，若干原來在0.10上執行正常的SQL會在新版上報錯誤 “return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask”，查看hive執行日志，從中找到如下錯誤

Total MapReduce jobs = 1 java.io.IOException: Cannot run program "/data/opt/hadoop_cdh5/bin/hadoop" (in directory "/root"): error=13, 權限不夠 at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041) at java.lang.Runtime.exec(Runtime.java:617) at java.lang.Runtime.exec(Runtime.java:450) at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.execute(MapredLocalTask.java:253) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1485) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1263) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1091) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.io.IOException: error=13, 權限不夠 at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:135) at java.lang.ProcessImpl.start(ProcessImpl.java:130) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022) ... 19 more FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask

從上邊錯誤及下邊報錯的類MapredLocalTask可以看出跟本地任務有關

hive從0.7版以后，為了提高小數據的計算速度，增加了本地模式，即將hdfs上的數據拉到hiveserver本地進行計算，可以通過以下幾個參數對相關行為進行設置

hive.exec.mode.local.auto=false
hive.exec.mode.local.auto.input.files.max=4
hive.exec.mode.local.auto.inputbytes.max=134217728
其中第一個為不啟用本地模式，第二個參數表示文件數小于4時使用本地模式，第三個參數表示文件大小小于128m時采用本地模式
默認為不啟用本地模式；在啟用的情況下，滿足第二、三個條件中的任意一個都會使用本地模式。
在之前我們用過的0.8.1、0.10版上都未遇到過上述錯誤，懷疑是現在0.12版本的問題突然導致上述錯誤。任務是在root用戶下通過crontab調用的，進入shell后先啟動hiveserver，所以默認工作目錄其實是/root；為了能正常讀寫hdfs上的文件，hiveserver在啟動時切換到了hdfs用戶，一旦遇到上述兩種滿足啟用本地模式的情況，hdfs用戶試圖向當前工作目錄/root拉取數據，必然沒有權限從而導致以上錯誤。
理清問題所在就好辦了，我們可以先創建一個目錄，把用戶、用戶組授權給hdfs，進入shell后，先切換工作目錄，然后再啟動hiveserver即可。如hdfs的home目錄/home/hdfs
然后在任務shell的公共配置文件conf/kettle.conf中增加一行切換目錄腳本即可解決以上問題
cd /home/hdfs

部分二 關于return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask 類似上一篇中return code 1的問題，這個也是跟hive本地任務有關系。

從hive的日志中可以找到出錯時本地日志文件，如下：

查看日志文件內容

2014-07-10 11:50:37,606 INFO mr.ExecDriver (SessionState.java:printInfo(417)) - Execution log at: /tmp/hdfs/hdfs_20140710114949_ab4d1d02-0637-4abd-9e45-2a27c5d740d9.log 2014-07-10 11:50:37,711 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/tmp/hdfs/hive_2014-07-10_11-49-37_877_2428431256361163465-1/-local-10009/jobconf 2014-07-10 11:50:37,720 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/tmp/hdfs/hive_2014-07-10_11-49-37_877_2428431256361163465-1/-local-10009/jobconf 2014-07-10 11:50:37,798 INFO log.PerfLogger (PerfLogger.java:PerfLogBegin(97)) - <PERFLOG method=deserializePlan from=org.apache.hadoop.hive.ql.exec.Utilities> 2014-07-10 11:50:37,798 INFO exec.Utilities (Utilities.java:deserializePlan(732)) - Deserializing MapredLocalWork via kryo 2014-07-10 11:50:38,043 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(124)) - </PERFLOG method=deserializePlan start=1404964237798 end=1404964238043 duration=245 from=org.apa 2014-07-10 11:50:38,050 INFO mr.MapredLocalTask (SessionState.java:printInfo(417)) - 2014-07-10 11:50:38 Starting to launch local task to process map join; maximum memory = 2014-07-10 11:50:38,059 INFO mr.MapredLocalTask (MapredLocalTask.java:initializeOperators(389)) - fetchoperator for t2:t_tmp_user_first_login created 2014-07-10 11:50:38,198 INFO exec.TableScanOperator (Operator.java:initialize(338)) - Initializing Self 0 TS 2014-07-10 11:50:38,198 INFO exec.TableScanOperator (Operator.java:initializeChildren(403)) - Operator 0 TS initialized 2014-07-10 11:50:38,199 INFO exec.TableScanOperator (Operator.java:initializeChildren(407)) - Initializing children of 0 TS 2014-07-10 11:50:38,199 INFO exec.SelectOperator (Operator.java:initialize(442)) - Initializing child 1 SEL 2014-07-10 11:50:38,199 INFO exec.SelectOperator (Operator.java:initialize(338)) - Initializing Self 1 SEL 2014-07-10 11:50:38,605 ERROR mr.MapredLocalTask (MapredLocalTask.java:executeFromChildJVM(324)) - Hive Runtime Error: Map local work failed java.lang.RuntimeException: java.lang.ClassNotFoundException: com.renren.hive.date.GetWeekISO at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.getUdfClass(GenericUDFBridge.java:132) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isStateful(FunctionRegistry.java:1474) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isDeterministic(FunctionRegistry.java:1437) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.isDeterministic(ExprNodeGenericFuncEvaluator.java:132) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.iterate(ExprNodeEvaluatorFactory.java:83) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.toCachedEval(ExprNodeEvaluatorFactory.java:73) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:59) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:453) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:409) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:188) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377) at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.initializeOperators(MapredLocalTask.java:408) at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:302) at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:728) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: java.lang.ClassNotFoundException: com.renren.hive.date.GetWeekISO at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:270) at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.getUdfClass(GenericUDFBridge.java:130) 由上可知，這次是找不到UDF的類（如遇到其他情況，需要具體問題具體分析），雖然在進入hive的時候通過add jar語句將自定義函數的jar包添加到hadoop集群，但在本地模式時確找不到了。定位到問題就好解決了：既然是local模式找不到udf jar包，說明在add jar步驟只是向當前job在hdfs上的工作目錄下添加了，無視本地工作目錄；那么我們就直接把udf的jar包copy到hive的lib目錄下，測 試正常。該問題在之前用過的hive 0.10、0.8.1中都未遇到過，初步猜測跟0.12版的bug有關，具體原因就需要花時間翻代碼對照前后版本的變動了

從org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask代碼中看，還有return code 3的情況，現在幸運的尚未遇到，遇到后再補記錄

總結

以上是生活随笔為你收集整理的hive异常 return code X from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask 解决的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。