Which is the best practice to define/handle BigQuery sharded tables like table_abc_* in Dataform?
Is there a way to reference them with ${ref(...)} and using a sort of _TABLE_SUFFIX syntax to select the one we need via variables? I was thinking to create a view on top of it selecting all the fields with the _table_suffix statement but this seems to me a workaround more than a way to define those in a proper way.
Defining Sharded BigQuery Tables on Dataform
305 Views Asked by Yak O'Poe At
1
There are 1 best solutions below
Related Questions in GOOGLE-BIGQUERY
- SQL LAG() function returning 0 for every row despite available previous rows
- Convert C# DateTime.Ticks to Bigquery DateTime Format
- SELECT AS STRUCT/VALUES
- Google Datastream errors on larger MySQL tables
- Can i add new label called looker-context-look_id in BigQuery connection(Looker)
- BigQuery external table using JSON files
- Does Apache Beam's BigQuery IO Support JSON Datatype Fields for Streaming Inserts?
- sample query for review for improvement on big query
- How does Big Query differentiate between a day and month when we upload any CSV or text file?
- How to get max value of a column when ids are unique but they are related through different variables
- how to do a filter from a table where 2 different columns has 2 different records which has same set of key combinations in bigquery?
- How to return a string that has a special character - BigQuery
- How do I merge multiple tables into a new table in BigQuery?
- Customer Churn Calculation
- Is it correct to add "UNNEST" in the "ON" condition of a (left) join?
Related Questions in DATAFORM
- How to run tasks which are after assertion in Google dataform even if assertion results in rows
- how do I executing JS loop returning multiple queries with Dataform
- How to restrict users of Google Dataform from pushing the committed code to the "default branch" instead of the "remote branch"
- How to convert DBT to Dataform
- How to setup Google Cloud Dataform git connection in a sub folder to use it in a monorepo?
- How to change the root definitions folder location for Google Cloud Dataform project?
- Syncfusion MAUI dataForm CheckBox, Date item's Label doesn't Appear in LeadingView
- Secrets Manager : GCP cost estimation
- How to set key value labels to BigQuery columns
- In GCP Dataform, how do I query a declared BQ table?
- How to use the dataform environments.json file when invocating the workflow from Airflow?
- Dataform Pre-operations Variables Not Executing in SQL Query
- Does dataform support CTE's or Subqueries?
- GCP Dataform - Can't connect repository to Github via SSH
- Use custom merge logic for incremental table in Dataform
Related Questions in GOOGLE-DATAFORM
- how execute a federated query in Google BigQuery with the federated query containing a variable
- Google Dataform incremental merge
- dataform schema suffix for sources
- Dataform reuse table confing in javascript
- Can we use promise in js blocks or .js files?
- The Default Dataform SA unable to query INFORMATION_SCHEMA.JOBS_BY_ORGANIZATION
- Remote GitHub repository cannot be reached: generic::invalid_argument in GCP
- Getting "Intl is not defined" in Dataform GCP project
- GCP Dataform Upstream source check
- Joining 2 tables and Optimization workflow using dataform
- Unable to connect to git hub repo from Dataform gcp
- Defining Sharded BigQuery Tables on Dataform
- Dataform - getting os username
- Execution of Google Dataform workflow gives error access denied
- Where is the repository stored that is created for Google Dataform?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular # Hahtags
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Quoting the answer I found on GitHub:
table_abc.sqlx
Then you can reference it in your other SQLX files as you wanted: