In GCP Dataform, how do I query a declared BQ table?

92 Views Asked by At

I have been following Google's Quickstart tutorial (https://cloud.google.com/dataform/docs/quickstart-create-workflow) to figure out how Dataform works, and in there we create a view:

config {
  type: "view"
}

SELECT
  "apples" AS fruit,
  2 AS count
UNION ALL
SELECT
  "oranges" AS fruit,
  5 AS count
UNION ALL
SELECT
  "pears" AS fruit,
  1 AS count
UNION ALL
SELECT
  "bananas" AS fruit,
  0 AS count

which can be accessed in a new .sqlx file by saying:

config {
    type: "view"
}

SELECT
  *
FROM
  ${ref("quickstart-source")}

However, now I have declared a source that points to an existing BQ table:

config {
    type: "declaration",
    database: "dataformtutorial-v2",
    schema: "raw_data_sources",
    name: "lws_data",
    description: "Raw data source"

}

but when I try to access it in the same way saying:

config {
    type: "view"
}

SELECT
  *
FROM
  ${ref("lws_data")}

I get an error: Could not resolve "lws_data". I have tried googling but am not finding any solutions so i assume my table declaration is incorrcet somehow or that I am missing a step. Any help would be appreciated.

1

There are 1 best solutions below

0
164_user On

I found the answer to my question, the issue was the location specified in the dataform.json vs the location of the BQ tables.

My BQ table was in europe-west 4 while the location of the workspace was EU (default). I thought this was fine since both is in Europe, but when I changed the workspace to europe-west 4 it worked.