eclipse提交mr到hadoop集群后任务卡住不执行
用eclipse远程提交mr到hadoop集群后,mr卡住后一直不执行,在控制台查日志发现错误如下:
org.apache.hadoop.ipc.Client: Retrying connect to server:127.0.0.1:8030. Already tried 0time(s).org.apache.hadoop.ipc.Client: Retrying connect to server:127.0.0.1:8030. Already tried 1time(s).org.apache.hadoop.ipc.Client: Retrying connect to server:127.0.0.1:8030. Already tried 2time(s).
果断把跟8030有关的端口配置到Configuration 里,问题解决,Configuration配置如下:
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://master190:9000/");
conf.set("mapreduce.framework.name", "yarn");
conf.set("yarn.resourcemanager.address", "master190:8032");
conf.set("yarn.resourcemanager.scheduler.address", "master190:8030");
conf.set("hbase.zookeeper.quorum", "master190,slave191,slave192,slave193");
注:错误原因是没有配置红色部分