欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

Could not parse Master URL: 'loacl[12]'

程序员文章站 2022-06-13 22:06:35
...

代码里设置了是spark本地运行,Master是“local[12]”,但是运行后说无法识别。出现过好几次,具体原因没有找到。

解决方案:重启idea后就好了。

  val conf = new SparkConf().setAppName("ALSCode2Recomm").setMaster("local[*]")

报错信息:

18/10/24 11:46:10 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
18/10/24 11:46:10 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
18/10/24 11:46:10 INFO Utils: Successfully started service 'SparkUI' on port 4042.
18/10/24 11:46:10 INFO SparkUI: Started SparkUI at http://192.168.56.1:4042
18/10/24 11:46:10 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Could not parse Master URL: 'loacl[12]'
	at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2735)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:522)
	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2281)
	at com.shandjj.itemCFRating$.<init>(itemCFRating.scala:17)
	at com.shandjj.itemCFRating$.<clinit>(itemCFRating.scala)
	at com.shandjj.itemCFRating.main(itemCFRating.scala)
18/10/24 11:46:10 INFO SparkUI: Stopped Spark web UI at http://192.168.56.1:4042
18/10/24 11:46:10 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
18/10/24 11:46:10 INFO MemoryStore: MemoryStore cleared
18/10/24 11:46:10 INFO BlockManager: BlockManager stopped
18/10/24 11:46:10 INFO BlockManagerMaster: BlockManagerMaster stopped
18/10/24 11:46:10 WARN MetricsSystem: Stopping a MetricsSystem that is not running
18/10/24 11:46:10 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
18/10/24 11:46:10 INFO SparkContext: Successfully stopped SparkContext
18/10/24 11:46:10 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
Exception in thread "main" java.lang.ExceptionInInitializerError
	at com.shandjj.itemCFRating.main(itemCFRating.scala)
Caused by: org.apache.spark.SparkException: Could not parse Master URL: 'loacl[12]'
	at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2735)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:522)
	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2281)
	at com.shandjj.itemCFRating$.<init>(itemCFRating.scala:17)
	at com.shandjj.itemCFRating$.<clinit>(itemCFRating.scala)
	... 1 more
18/10/24 11:46:10 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
18/10/24 11:46:10 INFO ShutdownHookManager: Shutdown hook called
18/10/24 11:46:10 INFO ShutdownHookManager: Deleting directory C:\Users\DELL\AppData\Local\Temp\spark-2e182f44-bfb3-4ce1-829f-1dccef17efef

Process finished with exit code 1

相关标签: spark