Maven安装(Linux)
程序员文章站
2022-08-30 08:10:09
1.从官网上寻找自己需要的合适的版本,此处我用的是maven-3.6.1 wget http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz 2.将其解压 ......
1.从官网上寻找自己需要的合适的版本,此处我用的是maven-3.6.1
wget http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz
2.将其解压在/usr/local 目录下
tar -zxvf apache-maven-3.6.1-bin.tar.gz -c /usr/local rm apache-maven-3.6.1-bin.tar.gz -c /usr/local
3.进入/usr/local目录下,修改maven文件目录的名字
cd /usr/local mv apache-maven-3.6.1 maven-3.6.1
4.接下来进行maven的环境配置
vim /etc/profile
export maven_home=/usr/local/maven-3.6.1 export path=$java_home/bin:$maven_home/bin:$path
5.刷新环境变量
source /etc/profile
6.测试maven是否成功安装
mvn -version
7.java独立应用编程
1)进入用户主文件夹
cd ~
2)创建sparkapp2
mkdir sparkapp2
3)创建一个simpleapp.java文件在./sparkapp2/src/main/java中
vim ./sparkapp2/src/main/java/simpleapp.java
内容如下:
1 /*** simpleapp.java ***/ 2 import org.apache.spark.api.java.*; 3 import org.apache.spark.api.java.function.function; 4 5 public class simpleapp { 6 public static void main(string[] args) { 7 string logfile = "file:///usr/local/spark-2.4.3/readme.md"; 8 javasparkcontext sc = new javasparkcontext("local", "simple app", 9 "file:///usr/local/spark-2.4.3/", new string[]{"target/simple-project-1.0.jar"}); 10 javardd<string> logdata = sc.textfile(logfile).cache(); 11 12 long numas = logdata.filter(new function<string, boolean>() { 13 public boolean call(string s) { 14 return s.contains("a"); 15 } 16 }).count(); 17 18 long numbs = logdata.filter(new function<string, boolean>() { 19 public boolean call(string s) { return s.contains("b"); } 20 }).count(); 21 22 system.out.println("lines with a: " + numas + ", lines with b: " + numbs); 23 } 24 }
该程序依赖spark java api,因此需要通过maven进行编译打包。
4)在./sparkapp2中新建文件pom.xml
此处的依赖如果不知道是哪个具体的版本,可以直接在maven的官网上查看依赖,官网链接
内容如下:
<project> <groupid>edu.berkeley</groupid> <artifactid>simple-project</artifactid> <modelversion>4.0.0</modelversion> <name>simple project</name> <packaging>jar</packaging> <version>1.0</version> <repositories> <repository> <id>akka repository</id> <url>http://repo.akka.io/releases</url> </repository> </repositories> <dependencies> <dependency> <!-- spark dependency --> <groupid>org.apache.spark</groupid> <artifactid>spark-core_2.11</artifactid> <version>2.4.3</version> </dependency> </dependencies> </project>
5)使用maven程序打包java程序
cd ~/sparkapp2 find .
6)接着将整个应用程序打包成jar(耗时长,十几二十分钟这样子吧,会比sbt时间要长),成功后的提示信息如下所示:
/usr/local/maven-3.6.1/bin.mvn package
7)通过spark-submit运行程序
将生成的jar包通过spark-submit提交到spark中
/usr/local/spark-2.4.3/bin/spark-submit --class "simpleapp" ~/sparkapp2/target/simple-project-1.0.jar 2>&1 | grep "lines with a"
上一篇: Jenkins实现Android自动打包并生成二维码
下一篇: 以返回值区分重载方法