Skip to content

BETWEEN @start_ds AND @end_ds results in an empty range #5689

@olegbbtr

Description

@olegbbtr

DLT generator generates INCREMENTAL_BY_TIME_RANGE models with the following WHERE suffix:

WHERE
  TO_TIMESTAMP(CAST(c._dlt_load_id AS DOUBLE)) BETWEEN @start_ds AND @end_ds

Problem: @start_ds and @end_ds are DATE values (without time), so when comparing against a TIMESTAMP column:

@end_ds = '2026-02-02' is interpreted as '2026-02-02 00:00:00'
Data from 2026-02-02 21:30:15 is excluded because 21:30:15 > 00:00:00

So if @start_ds = @end_ds, it results in an empty match.

I am not sure about the scope of this issue:

  1. Whether it only applies to the models generated by DLT integration.
  2. Whether it only applies for the duckdb dialect.

Suggestion on how to fix: instead of

BETWEEN @start_ds AND @end_ds

generate

BETWEEN @start_ts AND @end_ts

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions