SCD stage keep adding unchanged records into table

[abyss]

Hi all:
the SCD stage is keep adding unchanged records into table.
the job is quite simple, problem is this job keeping add records (always 41 records) into the table even there is no change on those records at all.
I use the hash partition on all the input and output link on this stage, but it doen't fix the problem.

41 rows insert into the table everytime i run the job. table should have 108 rows, none of them changes. some row get inserted so not.

can anyone help?

thanks in advance

[abyss]

and from the sequential output file, the content is same with the oracle lookup stage (ora_faculty_dim).
anyone know why?
and if SCD stage doesn't work, what other method i can use? change capture stage?

thanks
abyss

[naveenkumar.ssn]

hi Abyss,

you can do the SCD with the Change Data Capture,chksum and column export stages.

However before that can you please elaborate on whats your problem ?

With the post that you have given , we could not infer what is wrong ?

Regards
Naveen

[abyss]

thanks for your reply,
the problem is i compare main input data with existing data from oracle table by using 4 business keys. the main input data is exactly same with the data from dimension table, so there shouldn't be any update or insert into the dimension table. However there are 41 rows insert into the table every time i run the job.

cheers

[ArndW]

Problems like this can be tricky to sort out - I've just spent 2 days doing something similar with a SCD Type 2...

Can you reduce your data to just 1 row for test purposes, and write the update record to a peek or sequential file stage. Then compare that output column-for-column with the database contents to make sure that there is no changed value there after all? You should also ensure ensure that there are no implicit type conversions happening that might be falsifying your data contents.

[chulett]

Naveen, please stop quoting everything you are replying to all of the time. Do it selectively when it serves a purpose, sure, but all the time?

Instead please make use of the perfectly lovely Reply to topic link at the top and bottom of every discussion page. Your far too overworked DS Janitorial Service members will thank you.

[abyss]

Hi i somehow fix the problem.
the problem is the type of business keys is miss match. one is integer type and another one is decimal type. i am unsure why some records can match and some not.

thanks ArndW, your method certainly helps to identify the problem.

cheers