Evaluate expression of implicit TypeTag on Intellij debugger fails, why? and how can i enable it?

178 Views Asked by At
package spark.course.example

import org.apache.spark.sql.SparkSession

import org.apache.spark.sql.functions.typedLit

object Example extends App{

    val spark: SparkSession = SparkSession.builder.master("local[*]").appName("Example").getOrCreate

    import spark.implicits._

    val columns = Seq("language","users_count")

    val data = Seq(("Java", "20000"), ("Python", "100000"), ("Scala", "3000"))

    val df = spark.createDataFrame(data).toDF(columns:_*)

    val df_with_col = df.withColumn("raw",typedLit[Option[Int]](None))

}

I try do debug the last row and evaluate the expression of: df.withColumn("raw",typedLit[Option[Int]](None)) but getting error in the evalution that says Cannot find local variable 'TypeTag' Error evaluating method : 'TypeTag': Error evaluating method : 'TypeTag'

Why it can't find it? How can I make it work so i can debug & evaluate easily?

Thanks

Tried to add import scala.reflect.runtime.universe._; before the expression in the evaluation window and still it didn't find it

0

There are 0 best solutions below