hiveserver2服务异常停止,报错:java.lang.OutOfMemoryError: Java heap space
错误详情如下:
Exception in thread "HiveServer2-Handler-Pool: Thread-956" Exception in thread "HiveServer2-Handler-Pool: Thread-953" java.lang.OutOfMemoryError: Java heap space at java.util.HashMap.newNode(HashMap.java:1742) at java.util.HashMap.putVal(HashMap.java:630) at java.util.HashMap.put(HashMap.java:611) at org.apache.hive.service.rpc.thrift.TCLIService$Processor.getProcessMap(TCLIService.java:1353) at org.apache.hive.service.rpc.thrift.TCLIService$Processor.<init>(TCLIService.java:1330) at org.apache.hive.service.auth.TSetIpAddressProcessor.<init>(TSetIpAddressProcessor.java:48) at org.apache.hive.service.auth.PlainSaslHelper$SQLPlainProcessorFactory.getProcessor(PlainSaslHelper.java:151) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOfRange(Arrays.java:3664) at java.lang.String.<init>(String.java:207) at org.apache.xerces.xni.XMLString.toString(Unknown Source) at org.apache.xerces.parsers.AbstractDOMParser.characters(Unknown Source) at org.apache.xerces.xinclude.XIncludeHandler.characters(Unknown Source)错误原因:
由java.lang.OutOfMemoryError: Java heap space 可知 hiveserver2运行出现 JVM内存溢出
This Error happened during connect hiveserver2 via beeline, both happened from hiveserver node and remote node.
解决方法:
heapsize太小了,可以适当的调大些
Root cause, the heapsize of hadoop opts is too small, need to increase the size in hive-env.sh as bold.
修改hive-env.sh配置文件,详情修改内容如下
# Hive Client memory usage can be an issue if a large number of clients # are running at the same time. The flags below have been useful in # reducing memory usage: # if [ "$SERVICE" = "cli" ]; then if [ -z "$DEBUG" ]; then export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -Xmx4096m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:+UseParNewGC -XX:-UseGCOverheadLimit" else export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -Xmx4096m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:-UseGCOverheadLimit" fi fi # The heap size of the jvm stared by hive shell script can be controlled via: # export HADOOP_HEAPSIZE=4096 # # Larger heap size may be required when running queries over large number of files or partitions. # By default hive shell scripts use a heap size of 256 (MB). Larger heap size would also be # appropriate for hive server.
参考:
http://blog.csdn.net/lulynn/article/details/46455097
http://blog.csdn.net/levy_cui/article/details/53992402
声明:该文观点仅代表作者本人,牛骨文系教育信息发布平台,牛骨文仅提供信息存储空间服务。