Incremenatal loading reference data issue

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
pan
Participant
Posts: 3
Joined: Thu Mar 28, 2013 10:37 am
Location: oregon

Incremenatal loading reference data issue

Post by pan »

Hi
i am new to datastage ...
soure-sql server
target-oracle
datastage 8.7
unix environment

i am trying to load incrementally using chnage capture stage but i have a problem in refernece data(copy of target)..i am fetching letest data from source but after initial load when i tried to load the second load when the chnaged data records are zero(0)...the job deleting all the records from target...because it assuming no record coming from source(but all target records are looking up on source) so it is deleting all records from target..how can i avoid this without changing drop delete records to YES
because i need to extract delete records also


Thanks
prasannakumarkk
Participant
Posts: 117
Joined: Wed Feb 06, 2013 9:24 am
Location: Chennai,TN, India

Post by prasannakumarkk »

If it is a incremental load then the drop delete records should be set to zero.
because i need to extract delete records also
Which records you are mentioning about.
And you can have delete records seperated withhelp of a filter stage using the changeCode column in output link
Thanks,
Prasanna
pan
Participant
Posts: 3
Joined: Thu Mar 28, 2013 10:37 am
Location: oregon

Post by pan »

i need to fetch the inserted/updated/deleted records from source ..means those will come from chnage capture stage after comparing source and reference data(target data)....how can i do that incrementally
ds@tcs
Participant
Posts: 24
Joined: Thu Mar 17, 2011 6:26 am
Location: chennai
Contact:

Post by ds@tcs »

Capture the delete change code records into a dataset. in the next job do look up with source and delete from target.
Regards
Sankar
Post Reply