How can I write a source.yml in sqlmesh to import a table from Snowflake?

Question

I'm trying to run a model on sqlmesh just as in dbt but I'm finding some problems.First, I don't know what the syntax is for a source.yml in sqlmesh. I found examples on dbt but no example for sqlmesh. Also, I was wondering how can I organize my folders in sqlmesh as schemas in snowflake to save intermediate steps of the model. Again, I know what the code is for dbt but have no idea what it looks like for sqlmesh.

I tried using the same syntax in sqlmesh as in dbt but it does not work.

Answer 1

In SQLMesh, you can refer to your source tables directly in the models either as <schema>.<table_name> or <catalog>.<schema>.<tablename> if the source table is in a different catalog. Unlike dbt, no additional configuration information is required.

For example if I have a source table with columns cola and colb, schema foo, and table name bar, I could do the following directly in SQLMesh:

SELECT cola, colb FROM foo.bar

Additionally, if you want full column level lineage starting at the source tables, you can use the command sqlmesh create_external_models, which will go out to the datawarehouse and record the schema for all of the project's source tables locally in a file called schema.yaml. SQLMesh can do this because it is able to read and understand the SQL in your models, including which tables are source tables.