I am trying to use spring-cloud-dataflow to create streams over an existing set of servers. It is not a production environment, I am aware of the fault tolerance issues with the "local" deployer.
I have in the past created this infrastructure by:
1-Created a "scdf-server" by implementing the provided interface (ui and shell worked fine with no changes) . This server forward the request to an application (2) running on each server using REST calls
2-On each server I have an application that encapsulates the local deployer and receives the REST calls from the server passing the along to the encapsulated local deployer.
This approach worked perfectly, but now I need some of the new features in the new SCDF version and with the introduction of skipper I am not sure this is the best approach, or if it is even possible, need advice.
TIA
Looks like you have already explored the code to have better insights on how things are implemented.
As you stated, the
localdeployer implementation is never meant for production deployments. Hence, the SCDF team has dev-centric design goals onlocaldeployer implementation and also limited its scope on extensions as well.Given this, the way you designed your setup based on the needs, going with the Skipper server embedded into each app wouldn't work as Skipper server is expected to have a 1-on-1 relationship with SCDF server. Moreover, Skipper server also maintains a local State machine that might go stale when you have a different instance of Skipper server that listens to the same SCDF server. This makes the whole design very complex indeed.