Cannot resolve overloaded method 'createDataFrame' in scala code

30 Views Asked by At

I am trying to run the following scala code in Intellij IDE

'''

import org.apache.spark.graphx.GraphLoader
import org.apache.spark.sql.{SaveMode, SparkSession}

object FindTransMatch {
  def main(args: Array[String]): Unit = {
    println("Hello, World!")

        // Creates a SparkSession.
    val spark = SparkSession
      .builder
      .appName("FindTransMatch")
      .master("local")
      .getOrCreate()
    val sc = spark.sparkContext

    val graph = GraphLoader.edgeListFile(sc, args(0),false)

    // Find the connected profiles
    val cc = graph.connectedComponents().vertices
    spark
      .sqlContext
      .createDataFrame(cc.toJavaRDD())
      .write
      .mode(SaveMode.Overwrite).csv(args(1))
    spark.stop()
  }
}

'''

I get the error : Cannot resolve overloaded method 'createDataFrame'

0

There are 0 best solutions below