We currently store avro parquet files in S3. If I setup Apache Pinot to consume avro records from a kafka topic and provide query support, can I configure deep storage to offload this data to S3 in standard avro-parquet format in S3? Partitioned?
I've reviewed some code and read https://docs.pinot.apache.org/users/tutorials/use-s3-as-deep-store-for-pinot but I'm unable to tell if the off-loaded data could be used for historic queries through Redshift Spectrum or Athena as we can today.
Thanks!