I have a data in preprocessed structure stored in the database. I need to process it before it is ready for visualisation. The processing script can be hosted using lambda function or fargate api.
I plan on using Apache Superset as my visualisation tool. I wanted to configure a dashboard, where I can click a link which starts the processing (basically it just calls a url). Once the processing is complete, the script can add processed data to a different table in database, and then it can be visualised in Superset.
How can I do that? Also, if there are some other tools using which this can be done like Airflow, I am open to those. And if we need to use a different BI tool then Apache Superset, I am open to that as well.
Thanks for any help!