spark pom.xml配置
程序员文章站
2024-01-15 22:47:46
...
最近想利用spark dataframe 解决一些计算问题,由于版本众多,记录一下pom.xml的配置。以下配置,亲测通过:
<properties>
<spark.version>2.0.0</spark.version>
<scala.compat.version>2.11</scala.compat.version>
<scala.version>2.11.8</scala.version>
<jakson.version>2.6.0</jakson.version>
</properties>
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>${jakson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>${jakson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jakson.version}</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scala-lang.modules</groupId>
<artifactId>scala-xml_2.11</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.compat.version}</artifactId>
<version>${spark.version}</version>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_${scala.compat.version}</artifactId>
<version>${spark.version}</version>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependencies>
如果觉得本篇文章有所帮助,请帮忙微信扫描下本人写的一个实用小程序 “安全期随时看”,让你随时掌握爱爱时间,多谢帮助!
--------------------------- -------------------------
-------------------------------------------安全期随时看---------------------------------------------