I am using Apache Calcite to convert/parse a SQL String to SQLNode and parsing the SQLNode to translate it to Spark Dataset API's. I don't want to directly use Spark SQL parser to convert SQL to Spark SQL dialects.
What I am looking for is to use Calcite's parser and do the translation to Spark Java Dataset API. Currently the way I have been trying out is use Calcite Visitor interface to walk through - SqlSelect, SqlJoin, SQLIdentifier etc and translate that to Spark Dataset API's.
I just wanted advise on if there is a better way to do this. Will converting SqlNode to RelNode help ? Is there a way to convert RelNode to Spark Logical Plan and execute that logical Plan on Spark Engine? Does anyone have any ideas on how to do it? Any other ideas appreciated too!
Calcite doesn't currently support this, but it would make sense to include it in Calcite's Spark adapter if someone were to contribute it.
If you have time, please log a Calcite Jira case describing the requirements. You will not be committing yourself to implementing the feature, but it will help gather requirements, and will make it more likely that someone else will implement it.