Thank you for the reply.
I am using the default partitioning for both Sort and Filter - Auto.
And there are no NULLs.
Search found 21 matches
- Tue Nov 13, 2012 11:12 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Sort and Filter gives different results run to run
- Replies: 5
- Views: 3913
- Tue Nov 13, 2012 10:47 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Sort and Filter gives different results run to run
- Replies: 5
- Views: 3913
Sort and Filter gives different results run to run
I am performing a Sort and then Filter to remove duplicates. However I have noticed that with every run the results change by 1 record. The output from filter after the sort give 15 rows (where keychange=1) for run 1 and produces 2 records rejected i.e. (where keychange=0). But for Run 2, 3 records ...
- Mon Oct 10, 2011 4:15 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Efficient DS Job design to avoid errors
- Replies: 9
- Views: 4975
- Mon Oct 10, 2011 8:53 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Efficient DS Job design to avoid errors
- Replies: 9
- Views: 4975
- Sat Oct 08, 2011 10:03 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Efficient DS Job design to avoid errors
- Replies: 9
- Views: 4975
Efficient DS Job design to avoid errors
I have a job with nearly 100 stages and has multiple instance enabled and it runs 8x8. Until now it was working fine but today the job erred with the below message: APT_PMPlayer::APT_PMPlayer: fork() failed, Resource temporarily unavailable Seems that the above is due to resource contention issues. ...
- Thu Sep 30, 2010 8:39 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Function to check for strings
- Replies: 2
- Views: 1849
- Thu Sep 30, 2010 6:49 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Function to check for strings
- Replies: 2
- Views: 1849
Function to check for strings
I have input data which is being read as varchar and it has to b parsed out to get the zipcode.
We have a requirement which asks to check for strings in this field(zipcode).
How do I achieve this in the transformer?
We have a requirement which asks to check for strings in this field(zipcode).
How do I achieve this in the transformer?
- Thu Mar 11, 2010 12:36 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Handling Duplicates
- Replies: 2
- Views: 1871
Handling Duplicates
My input schema is Grp_ID Act_ID agtxid agtin And if I have a duplicate row then I use a sort sage to sort on all the 4 fields and set the key change column to true. So, in my transformer I want to put the word "MULTIPLE" if I see a duplicate record in a new output column. How can I achiev...
- Thu Mar 11, 2010 10:04 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Aggregating Data
- Replies: 3
- Views: 2194
Aggregating Data
I have 4 columns in my input as below Grp_ID Act_ID Agntin Agtxid And I want to count the number of rows that are duplicate across the 4 columns. So I am using an aggregate stage and am using the count no of rows to calculate the no of rows by grouping on all the 4 input columns. I am outputting the...
- Sat Feb 27, 2010 10:45 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Date Manipulation
- Replies: 7
- Views: 3363
- Sat Feb 27, 2010 9:29 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Date Manipulation
- Replies: 7
- Views: 3363
Date Manipulation
I have input data which is numeric and stored in 4 different fields as follows.
Century : 20
Year : 9
Month : 1
Day : 5
This data has to be formatted to arrive at the date in the format
2009-01-05 or 2009-01-05-HH.MM.SS.NNNNNN
How can I achieve this?
Century : 20
Year : 9
Month : 1
Day : 5
This data has to be formatted to arrive at the date in the format
2009-01-05 or 2009-01-05-HH.MM.SS.NNNNNN
How can I achieve this?
- Thu Feb 25, 2010 2:45 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Special Charcters
- Replies: 2
- Views: 1590
- Thu Feb 25, 2010 1:47 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Special Charcters
- Replies: 2
- Views: 1590
Special Charcters
I have an input column called the Telephone field which is varchar. It has data such as below: Telephone (999) 999 9999 999-999-9999 999 999 9999 999/9999999 999*9999999 How can I strip off all these special characters and just keep the numbers? I want a format like 9999999999. I know I can strip th...
- Thu Feb 04, 2010 3:46 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Aggregating Data
- Replies: 3
- Views: 2467
- Thu Feb 04, 2010 2:32 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Aggregating Data
- Replies: 3
- Views: 2467
Aggregating Data
I have 2 fields (id and name) such as Id Name 1 abc 2 def 3 ghi 1 jkl 2 mno 3 pqr 1 stu 2 vwx 3 123 1 456 2 789 3 111 The above records should be grouped into multiple batches with max batch size 3 records. And the combination of id and batchno should be unique. (Let's say I start my batch no.with 5...