Higher latency discrepancy in Spark application with Synapse ML package between jar execution and Docker containerization

Question

I have a Spark application leveraging the Synapse ML package for inferencing purposes. When I run the application as a standalone jar using java -jar my.jar, the p99 latency averages around 800ms.

Within the application, a server is initiated as illustrated below:

val df  = spark.readStream.server
  .address("0.0.0.0", 8898, "my_api")
  .option("name", "my_api")
  .load()

This code structure is based on the example provided in the official documentation:

https://microsoft.github.io/SynapseML/docs/Deploy%20Models/Quickstart%20-%20Deploying%20a%20Classifier/

However, upon Dockerizing the same code and exposing the port, I observe a significant spike in latency. Here's how I execute the application within the Docker container:

docker run -p 8898:8898 serveimagescala:0.0

Under this setup, the p99 latency escalates to approximately 2 seconds.

I'm puzzled by this considerable discrepancy and seek insights into potential factors contributing to the latency increase. Any suggestions or guidance on aspects I might be overlooking would be greatly appreciated.

Please feel free to request additional details if necessary. Thank you.