牛骨文教育服务平台(让学习变的简单)
博文笔记

执行hive内存溢出

创建时间:2013-10-31 投稿人: 浏览次数:145

错误日志:

Diagnostic Messages for this Task:

Error: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1703)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1563)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1401)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1333)




FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched: 
Job 0: Map: 10  Reduce: 3   Cumulative CPU: 3035.3 sec   HDFS Read: 2528366258 HDFS Write: 1209449 FAIL

Total MapReduce CPU Time Spent: 50 minutes 35 seconds 300 msec




解决方法:


Solution:
  Currently hive.map.aggr.hash.percentmemory is set to 0.5. Try setting it to a lower value. i.e "set hive.map.aggr.hash.percentmemory = 0.25;"
-----

声明:该文观点仅代表作者本人,牛骨文系教育信息发布平台,牛骨文仅提供信息存储空间服务。