欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

flume在windows10上运行

程序员文章站 2022-06-15 10:16:28
...

1. 直接下载官网flume1.8,解压到本地F盘

2. 在conf目录下复制flume-env.ps1.template改为flume-env.ps1,改下FLUME_CLASSPATH   

$FLUME_CLASSPATH="F:\kafka\apache-flume-1.8.0-bin\lib"

  flume-env.sh操作同上

3.检查安装是否成功:  bin\flume-ng.cmd version

  在查看version时,一直遇到如下错误:

F:\kafka\apache-flume-1.8.0-bin>bin\flume-ng.cmd version

F:\kafka\apache-flume-1.8.0-bin>powershell.exe -NoProfile -InputFormat none -ExecutionPolicy unrestricted -File F:\kafka\apache-flume-1.8.0-bin\bin\flume-ng.ps1 version

WARN: Config directory not set. Defaulting to F:\kafka\apache-flume-1.8.0-bin\conf
Sourcing environment configuration script F:\kafka\apache-flume-1.8.0-bin\conf\flume-env.ps1
Test-Path : 路径中具有非法字符。
所在位置 F:\kafka\apache-flume-1.8.0-bin\bin\flume-ng.ps1:106 字符: 56
+ ...                               ? { "$_" -ne "" -and (Test-Path $_ )} |
+                                                         ~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (F:\kafka\apache-flume-1.8.0-bin\":String) [Test-Path],ArgumentExceptio
    n
    + FullyQualifiedErrorId : ItemExistsArgumentError,Microsoft.PowerShell.Commands.TestPathCommand

conf/flume-env.ps1下的Test-Path字符非法,在该文件中ctrl+F,发现Test-Path共出现在三个地方:GetHadoopHome、GetHbaseHome、GetHiveHome。这可能与本机没有安装这三个服务有关吧,尝试注释掉这些代码行(360-405),再次运行后发现flume可以正常显示version了。如下:

F:\kafka\apache-flume-1.8.0-bin>bin\flume-ng.cmd version

F:\kafka\apache-flume-1.8.0-bin>powershell.exe -NoProfile -InputFormat none -ExecutionPolicy unrestricted -File F:\kafka\apache-flume-1.8.0-bin\bin\flume-ng.ps1 version

WARN: Config directory not set. Defaulting to F:\kafka\apache-flume-1.8.0-bin\conf
Sourcing environment configuration script F:\kafka\apache-flume-1.8.0-bin\conf\flume-env.ps1

  Running FLUME version :
    class: org.apache.flume.tools.VersionInfo
    arguments:

Flume 1.8.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
Revision: 99f591994468633fc6f8701c5fc53e0214b6da4f
Compiled by denes on Fri Sep 15 14:58:00 CEST 2017
From source with checksum fbb44c8c8fb63a49be0a59e27316833d

 

4. 运行flume 案例

  不加-Dflume.root.logger=INFO,console时可正常运行,但看不到输出的***,加后报如下错误:

F:\kafka\apache-flume-1.8.0-bin>bin\flume-ng.cmd agent -n agent -c conf -f conf\flume-conf.properties.template  -Dflume.root.logger=INFO,console

F:\kafka\apache-flume-1.8.0-bin>powershell.exe -NoProfile -InputFormat none -ExecutionPolicy unrestricted -File F:\kafka\apache-flume-1.8.0-bin\bin\flume-ng.ps1 agent -n agent -c conf -f conf\flume-conf.properties.template  -Dflume.root.logger=INFO,console
F:\kafka\apache-flume-1.8.0-bin\bin\flume-ng.ps1 : 找不到与参数名称“Dflume.root.logger=INFO,console”匹配的参数。
    + CategoryInfo          : InvalidArgument: (:) [flume-ng.ps1],ParentContainsErrorRecordException
    + FullyQualifiedErrorId : NamedParameterNotFound,flume-ng.ps1

  原来,在Windows上不能用Dflume这种设置(https://*.com/questions/35067940/any-idea-about-how-to-install-flume-with-hadoop-in-windows),应改为:

bin\flume-ng.cmd agent -n agent -c conf -f conf\flume-conf.properties.template -property  "flume.root.logger=INFO,console"

  运行后,可看到不断的***输出:

flume在windows10上运行