欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

Unable to load native-hadoop library for your platform解决

程序员文章站 2022-07-14 15:20:47
...

环境:

Eclipse: eclipse-jee-indigo-SR1-linux-gtk.tar.gz

Hadoop: hadoop-0.20.203.0

1、在使用Hadoop的压缩输出流时,发现总是报这隔错误:

Unable to load native-hadoop library for your platform

找了很多资料,都没能解决,最后终于发现是自己将文件名多加了个空格,导致本地库路径错误。

实际的路径是:

/home/hadoop/hadoop-0.20.203.0/lib/native/Linux-i386-32

另外要注意的是,Eclipse 运行配置的VM变量要加上一句:

-Djava.library.path=/home/hadoop/hadoop-0.20.203.0/lib/native/Linux-i386-32

Unable to load native-hadoop library for your platform解决

并在程序中输出该变量值,以保证设置正确:

		String libpath = System.getProperty("java.library.path");
		System.out.println("libpath=" + libpath);

2、如果用org.apache.hadoop.io.compress.DefaultCodec,则不需要lzo库,以免花费很多不必要的时间在编译安装lzo库上面,因为你可能根本用不着。

3、将日志设置为DEBUG级别,能获得更多的关于错误的信息。

	public static void main(String[] args) {		
		HdfsTest ht = new HdfsTest();
		String path = ht.getClass().getClassLoader().getResource("").getPath();
		System.out.println("path=" + path);
		
		String libpath = System.getProperty("java.library.path");
		System.out.println("libpath=" + libpath);
		
		PropertyConfigurator.configure(path + "log4j.properties");
                //do something
         }     

log4j.properties日志配置文件:

log4j.rootLogger=DEBUG,CONSOLE

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Threshold=DEBUG
log4j.appender.CONSOLE.Target=System.out
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d - %c -%-4r [%t] %-5p %c %x - %m%n

4、可能需要编译本地库,在RedHat AS4 上不编译本地库会报错:

[aaa@qq.com hadoop-0.20.203.0]$ more hs_err_pid8895.log
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGFPE (0x8) at pc=0x003d947c, pid=8895, tid=3074153392
#
# JRE version: 6.0_25-b06
# Java VM: Java HotSpot(TM) Server VM (20.0-b11 mixed mode linux-x86 )
# Problematic frame:
# C [ld-linux.so.2+0x847c]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

--------------- T H R E A D ---------------

Current thread (0x0805a000): JavaThread "main" [_thread_in_native, id=8921, sta
ck(0xb736d000,0xb73be000)]

siginfo:si_signo=SIGFPE: si_errno=0, si_code=1 (FPE_INTDIV), si_addr=0x003d947c

Registers:
EAX=0x0f4d007f, EBX=0x003e6fd4, ECX=0x00000000, EDX=0x00000000
ESP=0xb73bb02c, EBP=0xb73bb074, ESI=0x08304800, EDI=0x70984e4c
EIP=0x003d947c, EFLAGS=0x00010246, CR2=0x00000004

Top of Stack: (sp=0xb73bb02c)
0xb73bb02c: b73d7cce 70984619 b73bb0e8 b73bb000
0xb73bb03c: 00000000 00000000 00000000 70984e4c
0xb73bb04c: 70984618 70984218 00000007 08304b08
0xb73bb05c: 00000000 0f4d007f 70984619 003e6fd4
0xb73bb06c: 083049b0 70984619 b73bb148 003d9822
0xb73bb07c: 70984258 b73bb104 08304958 00000000
0xb73bb08c: 00000000 00000001 00000000 00000000
0xb73bb09c: 00511820 0000003c 003e6fd4 003e7650
. . .