So I have multiple streaming tables which can have an inflow of up to 10 million rows but right now I am trying to do a test in development stage and I have the 10 million records stored on a separate table in my hive metastore and I want to read that data into my streaming table but it takes as long as 10 mins for that data to be read.
Also, I am having issues performing things like windows functions and some other aggregations to compute kpis on my streaming table
Please any idea on how I can make it faster?