I think your best bet wud be either:
1)Use a hashed file stage or
2) Sort the records in Unix using sort command (before job sub-routine)
and then using stage variables in the transformer, reject the duplicate
records.The addition of Sort stage will surely slow down your job.
HTH
Search found 15 matches
- Sat May 13, 2006 12:24 am
- Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
- Topic: rejecting duplicate rows
- Replies: 5
- Views: 3031
- Sun Apr 23, 2006 11:00 pm
- Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
- Topic: How to handle excel worksheet as input
- Replies: 2
- Views: 1578
- Fri Apr 21, 2006 12:12 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: passing one column value to another if it is null
- Replies: 12
- Views: 4154
- Tue Jan 24, 2006 4:02 pm
- Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
- Topic: XML output
- Replies: 4
- Views: 2523
This doesn't seem to help us. The O/P we want: - <position> - <bondfwd> <Position_Name>ACGB Fwd</Position_Name> <Notional>660000</Notional> - <customBucketList> - <customBucket> <customDimensionName>Account_ID</customDimensionName> <customBucketValue>9369</customBucketValue> </customBucket> - <custo...
- Tue Jan 24, 2006 2:30 pm
- Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
- Topic: Proper XML format output
- Replies: 1
- Views: 1100
Proper XML format output
Hi I want to generate one xml file usinf datastasge 7.5 server job the file should look like - <position> - <bondfwd> <Position_Name>ACGB Fwd</Position_Name> <Notional>660000</Notional> - <customBucketList> - <customBucket> <customDimensionName>Account_ID</customDimensionName> <customBucketValue>936...
- Tue Jan 24, 2006 1:43 pm
- Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
- Topic: XML output
- Replies: 4
- Views: 2523
XML output
Hi, We have a requirement where we have to convert values from a DB table to XML.We are able to get the XML output but the format is not as required: O/P now:- <position> - <bondfwd> <Position_Name>ACGB Fwd 02-08-06 6.2500% 04-15-15</Position_Name> <Notional>660000</Notional> <Currancy>AUD</Currancy...
- Thu Jan 05, 2006 10:32 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: create column with null value in modify stage
- Replies: 4
- Views: 4872
- Tue Jan 03, 2006 11:12 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: diffrence between joiner and lookup
- Replies: 7
- Views: 4065
The main difference between joiner and lookup is in the wasy they handle the data and the reject links.In joiner, no reject links are possible.So we cannot get the rejected records directly.Lookup provides a reject link.Also lookup is used if the data being looked up can fit in the available tempora...
- Thu Dec 15, 2005 10:11 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Teradata Enterprise Stage vs Teradata Multiload Stage
- Replies: 11
- Views: 7489
Hi, The work tables are created each time you perform an operation using Multiload.It is automatically dropped once the job has run successfully.However if the job aborts, then the work tables have to be manully dropped before the job is run again.If your job has aborted, pl drop the work tables man...
- Mon Dec 12, 2005 10:36 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Teradata Enterprise Stage vs Teradata Multiload Stage
- Replies: 11
- Views: 7489
Multiload is very efficient when you are doing maintenance activities on multiple large tables.At a time Multiload stage can perform Inserts/Updates on upto 5 different tables in one pass. If you are using enterprise stage for this , then you will need five different stage instances of the Enterpris...
- Sat Dec 10, 2005 12:06 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Rows to columns
- Replies: 6
- Views: 2325
- Fri Dec 09, 2005 11:39 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Rows to columns
- Replies: 6
- Views: 2325
Rows to columns
Hi, How do you achieve the following in a parallel job: I have a source in the following format Id Name ----- --------- 1 xx 1 yy 1 zz 2 xx1 2 yy1 2 zz1 3 xx2 3 yy2 3 zz2 I want to get the data in the target as following id name1 name2 name3 ----- -------- ---------- ---------------- 1 xx yy zz 2 xx...
- Wed Nov 23, 2005 11:28 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Pad string with char fields
- Replies: 1
- Views: 3231
Re: Pad string with char fields
Hi Nick,
U can probably set the environment variable APT_STRING_PAD_CHAR to the char which you want to be used for padding instead of null.By default DS will pad string with null.HTH
U can probably set the environment variable APT_STRING_PAD_CHAR to the char which you want to be used for padding instead of null.By default DS will pad string with null.HTH
- Tue Jul 26, 2005 6:08 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Database polling in PX
- Replies: 3
- Views: 1498
Hi Siva, Thank You for you r reply. The polling interval in my case is about 20 secs as specified by the client. Suppose , I execute another job which writes to a file based on the polling result, will it create a dead lock situation since too many processes try to write to the same file (as the tim...
- Tue Jul 26, 2005 3:44 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Database polling in PX
- Replies: 3
- Views: 1498
Database polling in PX
How can I perform polling of a particular Oracle table continuously in PX 7.5 to find out any Inserts/Updates?