欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识！
您现在的位置是： 首页

eclipse提交mr(操作hbase)出错

程序员文章站 2024-01-23 16:35:34
...

通过eclipse 远程提交mr(处理文件到hbase)到hadoop集群报错：

错误信息如下：

2015-04-19 11:31:46,800 WARN  org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-04-19 11:31:47,753 INFO  org.apache.hadoop.conf.Configuration.deprecation - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-04-19 11:31:47,753 INFO  org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-04-19 11:31:47,821 INFO  org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2015-04-19 11:31:47,824 INFO  org.apache.hadoop.conf.Configuration.deprecation - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
2015-04-19 11:31:47,824 INFO  org.apache.hadoop.conf.Configuration.deprecation - mapred.job.name is deprecated. Instead, use mapreduce.job.name
2015-04-19 11:31:47,824 INFO  org.apache.hadoop.conf.Configuration.deprecation - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
2015-04-19 11:31:47,824 INFO  org.apache.hadoop.conf.Configuration.deprecation - mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
2015-04-19 11:31:47,824 INFO  org.apache.hadoop.conf.Configuration.deprecation - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2015-04-19 11:31:47,929 INFO  org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2015-04-19 11:31:47,929 INFO  org.apache.zookeeper.ZooKeeper - Client environment:host.name=lyq
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.6.0_45
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Sun Microsystems Inc.
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.home=D:\Program Files\Java\jdk1.6.0_45\jre
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=E:\Work\workspace\.metadata\.plugins\org.apache.hadoop.eclipse\hadoop-conf-401984830001496437;E:\Work\workspace\hadoop_study\bin;E:\Work\workspace\hadoop_study\lib\activation-1.1.jar;E:\Work\workspace\hadoop_study\lib\aopalliance-1.0.jar;E:\Work\workspace\hadoop_study\lib\asm-3.2.jar;E:\Work\workspace\hadoop_study\lib\avro-1.7.4.jar;E:\Work\workspace\hadoop_study\lib\commons-beanutils-1.7.0.jar;E:\Work\workspace\hadoop_study\lib\commons-beanutils-core-1.8.0.jar;E:\Work\workspace\hadoop_study\lib\commons-cli-1.2.jar;E:\Work\workspace\hadoop_study\lib\commons-codec-1.4.jar;E:\Work\workspace\hadoop_study\lib\commons-collections-3.2.1.jar;E:\Work\workspace\hadoop_study\lib\commons-compress-1.4.1.jar;E:\Work\workspace\hadoop_study\lib\commons-configuration-1.6.jar;E:\Work\workspace\hadoop_study\lib\commons-daemon-1.0.13.jar;E:\Work\workspace\hadoop_study\lib\commons-digester-1.8.jar;E:\Work\workspace\hadoop_study\lib\commons-el-1.0.jar;E:\Work\workspace\hadoop_study\lib\commons-httpclient-3.1.jar;E:\Work\workspace\hadoop_study\lib\commons-io-2.1.jar;E:\Work\workspace\hadoop_study\lib\commons-lang-2.5.jar;E:\Work\workspace\hadoop_study\lib\commons-logging-1.1.1.jar;E:\Work\workspace\hadoop_study\lib\commons-math-2.1.jar;E:\Work\workspace\hadoop_study\lib\commons-net-3.1.jar;E:\Work\workspace\hadoop_study\lib\guava-11.0.2.jar;E:\Work\workspace\hadoop_study\lib\guice-3.0.jar;E:\Work\workspace\hadoop_study\lib\guice-servlet-3.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-annotations-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-archives-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-auth-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-common-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-datajoin-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-distcp-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-extras-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-gridmix-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-hdfs-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-hdfs-nfs-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-app-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-common-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-core-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-hs-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-hs-plugins-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-jobclient-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-shuffle-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-examples-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-nfs-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-rumen-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-streaming-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-api-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-applications-distributedshell-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-client-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-common-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-server-common-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-server-nodemanager-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-server-resourcemanager-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-server-web-proxy-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-site-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hamcrest-core-1.1.jar;E:\Work\workspace\hadoop_study\lib\jackson-core-asl-1.8.8.jar;E:\Work\workspace\hadoop_study\lib\jackson-jaxrs-1.8.8.jar;E:\Work\workspace\hadoop_study\lib\jackson-mapper-asl-1.8.8.jar;E:\Work\workspace\hadoop_study\lib\jackson-xc-1.8.8.jar;E:\Work\workspace\hadoop_study\lib\jasper-compiler-5.5.23.jar;E:\Work\workspace\hadoop_study\lib\jasper-runtime-5.5.23.jar;E:\Work\workspace\hadoop_study\lib\javax.inject-1.jar;E:\Work\workspace\hadoop_study\lib\jaxb-api-2.2.2.jar;E:\Work\workspace\hadoop_study\lib\jaxb-impl-2.2.3-1.jar;E:\Work\workspace\hadoop_study\lib\jersey-core-1.9.jar;E:\Work\workspace\hadoop_study\lib\jersey-guice-1.9.jar;E:\Work\workspace\hadoop_study\lib\jersey-json-1.9.jar;E:\Work\workspace\hadoop_study\lib\jersey-server-1.9.jar;E:\Work\workspace\hadoop_study\lib\jets3t-0.6.1.jar;E:\Work\workspace\hadoop_study\lib\jettison-1.1.jar;E:\Work\workspace\hadoop_study\lib\jetty-6.1.26.jar;E:\Work\workspace\hadoop_study\lib\jetty-util-6.1.26.jar;E:\Work\workspace\hadoop_study\lib\jsch-0.1.42.jar;E:\Work\workspace\hadoop_study\lib\jsp-api-2.1.jar;E:\Work\workspace\hadoop_study\lib\jsr305-1.3.9.jar;E:\Work\workspace\hadoop_study\lib\junit-4.10.jar;E:\Work\workspace\hadoop_study\lib\log4j-1.2.17.jar;E:\Work\workspace\hadoop_study\lib\mockito-all-1.8.5.jar;E:\Work\workspace\hadoop_study\lib\netty-3.6.2.Final.jar;E:\Work\workspace\hadoop_study\lib\paranamer-2.3.jar;E:\Work\workspace\hadoop_study\lib\protobuf-java-2.5.0.jar;E:\Work\workspace\hadoop_study\lib\servlet-api-2.5.jar;E:\Work\workspace\hadoop_study\lib\slf4j-api-1.7.5.jar;E:\Work\workspace\hadoop_study\lib\slf4j-log4j12-1.7.5.jar;E:\Work\workspace\hadoop_study\lib\snappy-java-1.0.4.1.jar;E:\Work\workspace\hadoop_study\lib\stax-api-1.0.1.jar;E:\Work\workspace\hadoop_study\lib\xmlenc-0.52.jar;E:\Work\workspace\hadoop_study\lib\xz-1.0.jar;E:\Work\workspace\hadoop_study\lib\zookeeper-3.4.5.jar;E:\Work\workspace\hadoop_study\lib\hbase-client-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-common-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-examples-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-hadoop-compat-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-hadoop2-compat-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-it-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-prefix-tree-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-protocol-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-server-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-shell-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-testing-util-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-thrift-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\findbugs-annotations-1.3.9-1.jar;E:\Work\workspace\hadoop_study\lib\htrace-core-2.04.jar
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=D:\Program Files\Java\jdk1.6.0_45\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;D:/Program Files/Java/jdk1.6.0_45/bin/../jre/bin/server;D:/Program Files/Java/jdk1.6.0_45/bin/../jre/bin;D:/Program Files/Java/jdk1.6.0_45/bin/../jre/lib/amd64;D:\Program Files\Java\jdk1.6.0_45\bin;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;D:\App\hadoop-2.2.0\bin;D:\App\hadoop-2.2.0\sbin;D:\Program Files\SSH\;.;;D:\App\eclipse;;.
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=C:\Users\liuyq\AppData\Local\Temp\
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.name=Windows 7
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.version=6.1
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.name=liuyq
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.home=C:\Users\liuyq
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.dir=E:\Work\workspace\hadoop_study
2015-04-19 11:31:47,931 INFO  org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0x6a510e39, quorum=localhost:2181, baseZNode=/hbase
2015-04-19 11:31:47,956 INFO  org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process identifier=hconnection-0x6a510e39 connecting to ZooKeeper ensemble=localhost:2181
2015-04-19 11:31:48,111 INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:48,114 ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181
2015-04-19 11:31:48,149 WARN  org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
    at sun.nio.ch.Net.connect(Native Method)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
    at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266)
    at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276)
    at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
2015-04-19 11:31:48,559 INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:48,564 WARN  org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2015-04-19 11:31:48,564 INFO  org.apache.hadoop.hbase.util.RetryCounter - Sleeping 1000ms before retry #0...
2015-04-19 11:31:49,561 WARN  org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2015-04-19 11:31:49,662 WARN  org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2015-04-19 11:31:49,662 INFO  org.apache.hadoop.hbase.util.RetryCounter - Sleeping 2000ms before retry #1...
2015-04-19 11:31:50,665 INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:50,665 ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181
2015-04-19 11:31:50,665 WARN  org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
    at sun.nio.ch.Net.connect(Native Method)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
    at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266)
    at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276)
    at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
2015-04-19 11:31:50,768 INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:51,773 WARN  org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2015-04-19 11:31:51,874 WARN  org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2015-04-19 11:31:51,874 INFO  org.apache.hadoop.hbase.util.RetryCounter - Sleeping 4000ms before retry #2...
2015-04-19 11:31:52,874 INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:52,874 ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181
2015-04-19 11:31:52,874 WARN  org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
    at sun.nio.ch.Net.connect(Native Method)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
    at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266)
    at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276)
    at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)

 

 

原因：linux集群 etc/hosts 首行没有配置 127.0.0.1  localhost

解决方法：配置etc/hosts， 重启hadoop、hbase集群即可。



 


					

															

						相关标签：												
							hadoop						
												
							eclipse						
												
							hbase						
												
							127.0.0.1						
												
							0:0:0:0:0:0:0:1						
											

															

						

							上一篇：							
								eclipse提交mr到hadoop集群后任务卡住不执行							
						

						

							下一篇：							
								在集群上执行mr(操作hbase)任务报错							
						

					

															

						

							推荐阅读