I'm trying to write local timestamps to a Parquet file with Java. As far as I understand, Avro supports local timestamps (https://avro.apache.org/docs/1.11.0/spec.html), which corresponds to Parquet's TIMESTAMP(isAdjustedToUTC=false, unit=MILLIS . My schema definition looks like:
LocalTimestampMillis datetimeType = LogicalTypes.localTimestampMillis();
field = new Schema.Field(attrDesc.getAttributeName(), Schema.createUnion(datetimeType.addToSchema(Schema.create(Schema.Type.LONG)), Schema.create(Schema.Type.NULL)), null, null);
Apache Drill and duckdb interpret the field as Bigint / int64 and do not show the timestamp but just the integer value.
Did I miss something in the field definition or is this an incompatibility between libraries?
I'm using avro-1.11.1, parquet-avro-1.12.3, hadoop-core-1.2.1 and parquet-hadoop-1.12.3.