Skip to content

[Bug] No column lineage available using Postgres and DBT #5675

@mason-gordon

Description

@mason-gordon

Package Versions

sqlmesh - 0.228.2
psycopg2 - 2.9.11
dbt-core - 1.9.2
dbt-postgres -1.9.0

Issue

When I attempt to look at column lineage for any of my dbt models in the sqlmesh ui, there are no columns listed for any of my 250 models. The node graph shows model relationships, but no columns. When I trace it all the way back to our source models, they say "Unknown". I couldn't find anyone else having the same issue - no column lineage present when using sqlmesh over a dbt project, so I wonder what's unique about our configuration.

I followed the steps here: https://sqlmesh.readthedocs.io/en/stable/integrations/dbt/#installing-sqlmesh

  • I ran sqlmesh init -t dbt
  • I set a model start date of 2000-01-01
  • Ran sqlmesh plan and opted out of backfills (sqlmesh kept crashing trying to populate data)
  • I even tried sqlmesh create_external_models but the sqlmesh UI doesn't seem to recognize the models regardless.

All of my source models says that the source is Unknown.
Image

For whatever reason, there is column lineage for the dbt packages I have installed. I suspect that may be due to the fact the source tables are not created on the database:

Image

The Sqlmesh UI console shows a constant stream of api errors (adding text for searchability):

Request /api/files -> models/3_staging/odoo/stg_odoo__sale_order.sql with timeoutId: 216 timed out after 300000ms
Request /api/plan -> test with timeoutId: 62 timed out after 300000ms
Request /api/files with timeoutId: 43 timed out after 300000ms
Request /api/environments with timeoutId: 41 timed out after 300000ms
Request /api/models with timeoutId: 39 timed out after 300000ms
Request /api/modules with timeoutId: 6 timed out after 300000ms

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions