I'm currently evaluating the usage of Spring Cloud Data Flow for our infrastructure. We already use RabbitMQ and Kubernetes so that would be our target environment.
For local testing purposes I use dockerized MySQL and RabbitMQ and I want SCDF-Skipper to deploy the Stream-Services to my local docker instance so they can use the pre-existing MySQL and RabbitMQ-Containers (and I can manage and monitor everything in one single docker instance).
My first approach was to use Skipper and Dataflow Server from docker-compose but since I failed deploying something, I switched to use the jars following this tutorial:
https://dataflow.spring.io/docs/installation/local/manual/
By now, deployment of the stream works but fails to connect to my preexisting, dockerized MySQL. That is because by default SCDF Skipper seems to deploy to an internal Docker-Instance.
So my question is: Is there any way to configure SCDF Skipper to use the Docker-Instance on my local machine as deployment-target?
After another iteration of research, I stumbled upon https://dataflow.spring.io/docs/installation/local/docker/#docker-stream--task-applications
Apparently, to use Skipper and Dataflow-Server from within Docker (DooD, Docker-out-of-Docker), you have to add another docker-compose.yml.
That does NOT solve how to use a pre-existing docker-instance when running Skipper locally from jar, but at least it enables me to run them as a container on a pre-existing docker and thus lets it use it as deployment-target.