Importing CSV file but getting timestamp error

Question

I'm trying to import CSV files into BigQuery and on any of the hourly reports I attempt to upload it gives the code

Error while reading data, error message: Could not parse 4/12/2016 12:00:00 AM as TIMESTAMP for field SleepDay (position 1) starting at location 65 with message Invalid time zone: AM

I get that the format is trying to use AM as a timezone and causing an error but I'm not sure how best to work around it. All of the hourly entries will have AM or PM after the date-time and that will be thousands of entries.

I'm using the autodetect for my schema and I believe that's where the issue is coming up, but I'm not sure what to put in the edit as text schema option to fix it

Answer 1

To successfully parse an imported string to timestamp in Bigquery, the string must be in the ISO 8601 format.

YYYY-MM-DDThh:mm:ss.sss

If your source data is not available in this format, then try the below approach.

1. Import the CSV into a temporary table by providing explicit schema, where timestamp fields are strings.

2. Select the data from the created temporary table, use the BigQuery PARSE_TIMESTAMP function as specified below and write to the permanent table.

INSERT INTO `example_project.example_dataset.permanent_table`
SELECT
    PARSE_TIMESTAMP('%m/%d/%Y %H:%M:%S %p',time_stamp) as time_stamp,
    value
FROM `example_project.example_dataset.temporary_table`;