I'm new to Dremio, and I was following this SQS + S3 + Dremiotutorial to learn more about Dremio. In one of the code snippets, it is mentioned that get_messages_from_queue will create a CSV file and which is later used in the upload_file method to upload into S3.
However I'm missing that portion of the command which converts into CSV, can anyone help me how to create CSV using pandas? I'm new to Pandas, still learning.
SQS message body looks like this
"Body": "{\"holiday\":\"None\",\"temp\":288.28,\"rain_1h\":0.0,\"snow_1h\":0.0,\"clouds_all\":40,\"weather_main\":\"Clouds\",\"weather_description\":\"scattered clouds\",\"date_time\":\"2012-10-02 09:00:00\",\"traffic_volume\":5545}"
Add sqs message to dictionary and load it to panda
Dataframe. Finally useto_csvto export csv file.