I am having an issue with Athena having used AWS Glue to crawl an S3 bucket and create a table. The table created had a column type which was defined as a struct. I manually changed the type to String in the definition as I wish to use the contents by casting it to JSON.
However when I attempt to quert the data in Athena I receive
HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced.
This is because the partitions created automatically from the crawler refer to the changed column type as its old type (not the updated string type). I can't see how I can edit the partitions but also there are thousands of them as one is generated per folder so doing so manually could be difficult.
Does anyone have a way of editing a table scheme that also updates the partitions generated for it?
Thanks.