Getting error, when scan the data from unity catalog to purview

48 Views Asked by At

when I scan the data from unity, default scan rules are coming and selecting automatically, enter image description here and because of the scan rules my scanning completed with errors. enter image description here But data ingestion is ok. error is coming from billing.usage table because of date format: Timestamp,ErrorCode,OperationName,OperationItem,Message2024-03-08T12:26:46.350Z,FailToConnect,ReadData,databricks://adb-5159569612410553.13.azuredatabricks.net/da981d27-d47a-4a28-bc27-21c0f8094eea/dpwp_dev/system_logging/billing_usage,"[Databricks]DatabricksJDBCDriver ERROR processing query/statement. Error Code: 0, SQL state: org.apache.hive.service.cli.HiveSQLException: Error running query: [CANNOT_UP_CAST_DATATYPE] org.apache.spark.sql.AnalysisException: [CANNOT_UP_CAST_DATATYPE] Cannot up cast system.billing.usage.usage_metadata from ""STRUCT<cluster_id: STRING, job_id: STRING, warehouse_id: STRING, instance_pool_id: STRING, node_type: STRING, job_run_id: STRING, notebook_id: STRING, dlt_pipeline_id: STRING>"" to ""STRUCT<cluster_id: STRING, job_id: STRING, warehouse_id: STRING, instance_pool_id: STRING, node_type: STRING, job_run_id: STRING, notebook_id: STRING>""."

But I do not want to select a scan rule set or I would like to make an exemption, if there is no solution for that which scan rule i can set?

I created a custom scan rule set which has no rule: enter image description here and applied this scan, I got same error.

0

There are 0 best solutions below