Could not parse Master URL: 'loacl[12]'
程序员文章站
2022-06-13 22:06:35
...
代码里设置了是spark本地运行,Master是“local[12]”,但是运行后说无法识别。出现过好几次,具体原因没有找到。
解决方案:重启idea后就好了。
val conf = new SparkConf().setAppName("ALSCode2Recomm").setMaster("local[*]")
报错信息:
18/10/24 11:46:10 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
18/10/24 11:46:10 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
18/10/24 11:46:10 INFO Utils: Successfully started service 'SparkUI' on port 4042.
18/10/24 11:46:10 INFO SparkUI: Started SparkUI at http://192.168.56.1:4042
18/10/24 11:46:10 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Could not parse Master URL: 'loacl[12]'
at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2735)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:522)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2281)
at com.shandjj.itemCFRating$.<init>(itemCFRating.scala:17)
at com.shandjj.itemCFRating$.<clinit>(itemCFRating.scala)
at com.shandjj.itemCFRating.main(itemCFRating.scala)
18/10/24 11:46:10 INFO SparkUI: Stopped Spark web UI at http://192.168.56.1:4042
18/10/24 11:46:10 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
18/10/24 11:46:10 INFO MemoryStore: MemoryStore cleared
18/10/24 11:46:10 INFO BlockManager: BlockManager stopped
18/10/24 11:46:10 INFO BlockManagerMaster: BlockManagerMaster stopped
18/10/24 11:46:10 WARN MetricsSystem: Stopping a MetricsSystem that is not running
18/10/24 11:46:10 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
18/10/24 11:46:10 INFO SparkContext: Successfully stopped SparkContext
18/10/24 11:46:10 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.shandjj.itemCFRating.main(itemCFRating.scala)
Caused by: org.apache.spark.SparkException: Could not parse Master URL: 'loacl[12]'
at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2735)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:522)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2281)
at com.shandjj.itemCFRating$.<init>(itemCFRating.scala:17)
at com.shandjj.itemCFRating$.<clinit>(itemCFRating.scala)
... 1 more
18/10/24 11:46:10 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
18/10/24 11:46:10 INFO ShutdownHookManager: Shutdown hook called
18/10/24 11:46:10 INFO ShutdownHookManager: Deleting directory C:\Users\DELL\AppData\Local\Temp\spark-2e182f44-bfb3-4ce1-829f-1dccef17efef
Process finished with exit code 1
上一篇: 你是在为烤焦那只鸡找借口