Scala - How to compile code from an external file at runtime?

Question

I want to design a Scala program that accepts Scala files as parameters which can customize the execution of the program. In particular, I want to supply at runtime files that contain implementations of methods that will be invoked by the program. How can I properly depend on external files and invoke their methods dynamically at runtime? Ideally, I would also like those files to be able to depend on methods and classes in my program.

Example Scenario: I have a function that contains the line val p: Plant = Greenhouse.getPlant(), and the Greenhouse class with the getPlant method is defined in one of the files that will be supplied at runtime. In that file, the method getPlant returns a Rose, where Rose <: Plant and Plant is defined in the original program. How do I achieve (or approximate) this interdependency, assuming the files are only joined at runtime and not at compile-time?

Answer 1

Here's how to do it using only standard Scala. The non-obvious stuff is all in GreenhouseFactory:

package customizable

abstract class Plant

case class Rose() extends Plant

abstract class Greenhouse {
  def getPlant(): Plant
}

case class GreenhouseFactory(implFilename: String) {
  import reflect.runtime.currentMirror
  import tools.reflect.ToolBox
  val toolbox = currentMirror.mkToolBox()
  import toolbox.u._
  import io.Source

  val fileContents = Source.fromFile(implFilename).getLines.mkString("\n")
  val tree = toolbox.parse("import customizable._; " + fileContents)
  val compiledCode = toolbox.compile(tree)

  def make(): Greenhouse = compiledCode().asInstanceOf[Greenhouse]
}

object Main {
  def main(args: Array[String]) {
    val greenhouseFactory = GreenhouseFactory("external.scala")
    val greenhouse = greenhouseFactory.make()
    val p = greenhouse.getPlant()

    println(p)
  }
}

Put your override expression in external.scala:

new Greenhouse {
  override def getPlant() = new Rose()
}

The output is:

Rose()

The only tricky thing is that GreenhouseFactory needs to prepend that import statement to provide access to all the types and symbols needed by the external files. To make that easy, make a single package with all those things.

The compiler ToolBox is sort of documented here. The only thing you really need to know, other than the weird imports, is that toolbox.parse converts a string (Scala source code) into an abstract syntax tree, and toolbox.compile converts the abstract syntax tree into a function with signature () => Any. Since this is dynamically compiled code, you have to live with casting the Any to the type that you expect.

Answer 2

Scala doesn't provide this sort of functionality natively. The easiest way I know of to do this is with the Twitter "util-eval" library. This library wraps the necessary calls to the Scala compiler and the various class loading rituals, saving you enormous amounts of effort. The calls sequence to do what you're describing would look something like

val eval = new Eval()
val greenhouse = eval.apply[Greenhouse](new File("path/to/MyGreenhouse.scala"))
val plant = greenhouse.getPlant()

Your dynamically loaded Scala file needs to contain an expression, not a class per se, but that's pretty easy to do, basically like this.

new Greenhouse{
    def getPlant() = //thing to return the plant 
}

As I understand it, Twitter uses (or at least used) this functionality to have it's configuration files as Scala rather than properties/json/xml.