SBT Provided dependencies are not excluded with assembly

79 Views Asked by At

I am new to sbt. I have a quite simple project config as follow:

ThisBuild / version := "0.1.0-SNAPSHOT"

ThisBuild / scalaVersion := "2.12.15"

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", _*) => MergeStrategy.discard
  case _ => MergeStrategy.first
}

lazy val root = (project in file("."))
  .settings(
    name := "sifi_pipeline"
  )
libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "3.3.2" % Provided,
  "org.apache.spark" %% "spark-sql" % "3.3.2" % Provided,
  "org.apache.spark" %% "spark-streaming" % "3.3.2" % Provided,
  "org.apache.spark" %% "spark-sql-kafka-0-10" % "3.3.2",
  "com.google.cloud.bigdataoss" % "gcs-connector" % "1.4.3-hadoop2"
)

In my understanding provided scope means you make sure such dependencies are provided in class path and will NOT be include in the fat jar to be generated.

Here I have 5 dependencies, no matter how I config the provided scope. The assembly result were same in size. I even tried to add provided to all of them, still the same. Do I misunderstand provided?

0

There are 0 best solutions below