欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

spark pom.xml配置

程序员文章站 2024-01-15 22:47:46
...

最近想利用spark dataframe 解决一些计算问题,由于版本众多,记录一下pom.xml的配置。以下配置,亲测通过:

<properties>
	<spark.version>2.0.0</spark.version>
	<scala.compat.version>2.11</scala.compat.version>
	<scala.version>2.11.8</scala.version>
	<jakson.version>2.6.0</jakson.version>
</properties>
<dependencies>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-annotations</artifactId>
			<version>${jakson.version}</version>
		</dependency>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-core</artifactId>
			<version>${jakson.version}</version>
		</dependency>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-databind</artifactId>
			<version>${jakson.version}</version>
		</dependency>
		<dependency>
			<groupId>org.scala-lang</groupId>
			<artifactId>scala-library</artifactId>
			<version>${scala.version}</version>
		</dependency>
		<dependency>
			<groupId>org.scala-lang.modules</groupId>
			<artifactId>scala-xml_2.11</artifactId>
			<version>1.1.1</version>
		</dependency>
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-core_${scala.compat.version}</artifactId>
			<version>${spark.version}</version>
			<exclusions>
				<exclusion>
					<groupId>com.fasterxml.jackson.core</groupId>
					<artifactId>jackson-databind</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-sql_${scala.compat.version}</artifactId>
			<version>${spark.version}</version>
			<exclusions>
				<exclusion>
					<groupId>com.fasterxml.jackson.core</groupId>
					<artifactId>jackson-databind</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
<dependencies>		

如果觉得本篇文章有所帮助,请帮忙微信扫描下本人写的一个实用小程序 “安全期随时看”,让你随时掌握爱爱时间,多谢帮助!
---------------------------spark pom.xml配置 -------------------------
-------------------------------------------安全期随时看---------------------------------------------