I have case classes like these
case class PrimeraRemoteCopyConfig(
links: Option[Vector[JsObject]] = None,
status: Option[Vector[JsObject]] = None,
targets: Option[Vector[JsObject]] = None,
groups: Option[Vector[JsObject]] = None,
groupTargets: Option[Vector[JsObject]] = None,
groupVolumes: Option[Vector[JsObject]] = None)
object PrimeraRemoteCopyConfig {
implicit val _format = Json.format[PrimeraRemoteCopyConfig]
}
case class PrimeraConfig(
systemUid: String,
tenantId: String,
systemWWN: String,
remoteCopyConfig: Option[PrimeraRemoteCopyConfig] = None)
object PrimeraConfig {
implicit val _format = Json.format[PrimeraConfig]
}
And I have spark dataset that uses state management, using flatMapGroupsWithState.
However I am getting
22/05/10 17:07:48 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, 192.168.0.199, 53277, None)
Exception in thread "main" java.lang.UnsupportedOperationException: No Encoder found for play.api.libs.json.JsValue - map value class: "play.api.libs.json.JsValue"
- field (class: "scala.collection.Map", name: "underlying")
- array element class: "play.api.libs.json.JsObject"
- option value class: "scala.collection.immutable.Vector"
- field (class: "scala.Option", name: "links")
- option value class: "model.PrimeraRemoteCopyConfig"
- field (class: "scala.Option", name: "remoteCopyConfig")
- root class: "model.PrimeraConfig"
at org.apache.spark.sql.errors.QueryExecutionErrors$.cannotFindEncoderForTypeError(QueryExecutionErrors.scala:1000)
I tried defining encoders for JsObject using Encoders.product[JsObject] However it does not work. What am I doing wrongly here