Java spark example runs wrong with error: java.lang.NoSuchMethodError: scala.Predef$.$scope()Lscala/xml/TopScope$

Question

I wrote a example with spark maven support in Intelligent IDEA. The spark version is 2.0.0, hadoop version is 2.7.3, scala version is 2.11.8. Enviroment in system and IDE is the same version. Then application runs with error:

Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.$scope()Lscala/xml/TopScope$; at org.apache.spark.ui.jobs.StagePage.(StagePage.scala:44) at org.apache.spark.ui.jobs.StagesTab.(StagesTab.scala:34) at org.apache.spark.ui.SparkUI.(SparkUI.scala:62) at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:215) at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:157) at org.apache.spark.SparkContext.<init>(SparkContext.scala:443) at org.apache.spark.SparkContext.<init>(SparkContext.scala:149) at org.apache.spark.SparkContext.<init>(SparkContext.scala:185) at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:92) at com.spark.test.WordCountTest.main(WordCountTest.java:25) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

Answer 1

Spark 2.0.0 build with scala 2.10, you have to add scala 2.10 as framework support

Answer 2

Update pom.xml with scala 2.11.8.
See the spark download documentation for version compatibility.