DSXchange

Thanks but -
Require generic approach - 9 space would be specific to the metedata here in input but the input dataset can have varied number of columns and varying metadata.

I have a job wherein the RunTime Column Propagation is enabled. The job is designed as under: Dataset -----> Fileset. The Dataset, Fileset names are path passed as parameters. And the metadata is propagated at runtime using RCP. Now the Dataset passed have four columns A (Integer 5) B (Integer 5) C ...

There is a DataStage job which got locked. While trying to compile the job I am getting the error as " Cannot get access to the jobs " We are unable to either compile or delete the jobs. Also there being no locks in the job and DS Admin being trying to get that unlocked and still not able ...

Agregator was tested but it fills up the scratchdisk and the records are sorted on the key fields.

Thanks
Abhijit

Input - Col_A|Col_B|Col_C|Col_Date1|Col_Date2 Its 5 different columns delimited with pipe for illustration.... What I did is - for Col_Date1.... current = Col_A:Col_B:Col_C currentDt1=Col_Date1 chkDt1=if current=previous and Col_Date1>prev_Col_Date1 then prev_Col_Date1 else Col_Date1 flag = if curre...

"Col_A|Col_B|Col_C|Col_Date1|Col_Date2", these being the columns in the output but aggregation on 2 million rows would be slow. In stage veriable would it be possible if i sort by the keys Col_A|Col_B|Col_C ... only and then compare the Col_Date1 values and store in another stage variable ...

Thanks a lot for the elaborated reply.... But as the number of records being too large can the three sort stage be reduced to one and sorting done on keys and the stage variable taking care of the earliest and latest dates..... Else was looking for some more optimization looking at the large number ...

The input data looks somethin like - Col_A|Col_B|Col_C|Col_Date1|Col_Date2 A|B|C|2001/01/01|2002/01/03 A|B|C|2001/01/02|2002/01/04 A|B|C|2001/01/03|2002/01/05 The output should be: Col_A|Col_B|Col_C|Col_Date1|Col_Date2 A|B|C|2001/01/01|2002/01/05 Requirement: if Col_A, Col_B and Col_C are same among...

Requirement: The input data looks somethin like - Col_A|Col_B|Col_C|Col_Date1|Col_Date2 A|B|C|2001/01/01|2002/01/03 A|B|C|2001/01/02|2002/01/04 A|B|C|2001/01/03|2002/01/05 The output should be: Col_A|Col_B|Col_C|Col_Date1|Col_Date2 A|B|C|2001/01/01|2002/01/05 Requirement: if Col_A, Col_B and Col_C a...

Its simple look up with entire partitioning ..... and the job sometimes handles about 50M rows in the input link so its runtime gets around 3 hours .....

So please suggest some solution

I have a DataStage Job which uses lookup stage to lookup on some key values coming from two links. The amount of incoming records being quite high around 50 millions. So the job performance gets badly hampered. Need suggestion from the group in the ways the jobs performance can be improved and which...

The job uses parameter set ps_List passed to the job in form of list having values as.... ps_List1 ps_List2 Script calls the DS Job as under: sh TestScript.sh ps_List Now when I call the script from another DS job and pass the default value for the parameter set it takes the value of the parameter s...

Any response from anyone on this please.
Thanks
Abhijit

I want to call a DS job from an Unix shell script DS Job name: TestJob Script Name : TestScript.sh The job uses parameter set ps_List passed to the job in form of list having values as.... ps_List1 ps_List2 Script calls the DS Job as under: sh TestScript.sh ps_List Now when I call the script from an...

If the job aborts after loading 600 records out of 1000 in source file .... verify if the records 600 are actually loaded. Then in the output database stage whatever it be use UPSERT Update then Insert in place of simple insert and do not forget to specify the KEY fields in that. That will do yours ...