Change Capture Stage passing columns

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
BREIDY
Participant
Posts: 4
Joined: Mon Feb 13, 2006 12:09 pm

Change Capture Stage passing columns

Post by BREIDY »

I am using a Change Capture stage and would like for it to pass through a column from the Before dataset. I am not finding where this is possible the only values that are made available in the Output tab are from the After dataset.

Example:
Before dataset
Key_col
Value_col
Value2_col
Value3_col
Timestamp_col

After dataset
Key_col
Value_col
Value2_col
Value3_col

Desired return set:
Key_col (from after)
Value_col (from after)
Value2_col (from after)
Value3_col (from after)
Timestamp_col (from before)
Mat01
Premium Member
Premium Member
Posts: 50
Joined: Wed Jun 02, 2004 11:12 am
Location: Montreal, Canada

Post by Mat01 »

Hi Breidy,

viewtopic.php?t=94943&highlight=change+capture

Look for the last post

HTH,

Mat
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Change Apply can be used in conjuction with Change Capture to obtained the Before/After Fields.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
BREIDY
Participant
Posts: 4
Joined: Mon Feb 13, 2006 12:09 pm

Post by BREIDY »

I am not getting the Change Apply to working that way. I am not able to retreive any fields from the before record set only the changed recordset.

Additionally I want to use the change_code values to determine various date fields (Create, Update, Delete dates), not to acutally perform any changes. So if change_code = 1 then I set up the Create date else it stays equal to what should be returned from the before dataset and if change_code = 2 then Delete date gets set (all other dates remain same from before dataset) and records is NOT deleted.

I was able to accomplish this using a merge stage and it worked great for the first job I set up, but now with a new table (much larger dataset both rows and columns) I am getting errors stating that the input fields do not exist.

So my job is Before_Data and After_Data into Change_Capture stage, then Change_Capture stage and Before_Data into Merge stage into Transformer Stage into Oracle table
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Yes you got the way out. You can use merge/Join - Transformer for this purpose. But shouldnt be getting any erros like "input fields do not exist. "
Pls let know the number of columns you use.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Post Reply