Pentaho PDI SCD Type 2 Inserts new row when data has not changed

Question

I'm starting to use Pentaho PDI Version 9.3 for experimenting with Type 2 SCD. But when I run the same transformation 2 times, with the same data (no change in data), a new version of each row gets inserted every time, even though the row data has not changed. This is my setup:

Overall view:

Dimension Lookup/Update Setup - Keys

Dimension Lookup/Update Setup - Fields

Expected outcome No matter how many times I run this, if the values of exercise and short_name have not changed, no new rows should be added. But when I

Actual outcome A new version of each and every record is created each time I run the transformation, even when the exercise and short_name fields have not changed.

Answer 1

I had the exact same issue recently, and we eventually tracked the source to the fact one of the columns in the source table (queried by a Table Input step like yours) had a different data type than the corresponding column in the target table (also updated by a Dimension Lookup/Update step like yours). (It could also be you're casting the values in your Select values step, which I imagine causes the same behaviour.)