Search found 45 matches

by jiegao
Wed Oct 03, 2012 10:53 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Calculate the difference in seconds for two timestamps field
Replies: 3
Views: 2667

Calculate the difference in seconds for two timestamps field

Hi.
I am wondering what is the best way to calcuate the difference in seconds for two timestamps fields in Server job. Thanks in advance.
by jiegao
Tue Jul 10, 2012 11:54 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Should I use checksum
Replies: 5
Views: 2950

Thanks Craig
by jiegao
Tue Jul 10, 2012 5:16 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Should I use checksum
Replies: 5
Views: 2950

Thanks Craig. I am not planning to store the checksum on the table record. Does it make sense to calculate both checksum and compare? I am concerned more on the performance. I plan to just compare the concatenated value. What do you think?
by jiegao
Mon Jul 09, 2012 9:50 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Should I use checksum
Replies: 5
Views: 2950

Should I use checksum

I need to compare a table row with a row in a file to see if any field has changed. I am thinking to concatenation required fields and do the comparison. I am not sure if I should calculate the checksum then do the comparison. Which one is more efficient? Thank you in advance.
by jiegao
Fri Mar 09, 2012 12:15 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Table Column Definition
Replies: 5
Views: 2987

Thanks
by jiegao
Thu Mar 08, 2012 1:35 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Table Column Definition
Replies: 5
Views: 2987

Table Column Definition

Hello, One of the column of Oracle table changes from number(5,2) to number(10,2). Should I change the table definition accordingly in data stage? I have a feeling that data stage does not need to change. Since I do not have access to data stage currently, I can not test it out. Can some one tell me...
by jiegao
Fri Nov 25, 2011 8:24 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Parallel routines and emtpy strings
Replies: 5
Views: 4561

PhilHibbs, is there a specific reason you're passing to pxStrFirstCharList by reference and not by value?
by jiegao
Thu Nov 17, 2011 8:33 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Join growth on distinct Account ID
Replies: 1
Views: 1481

Re: Join growth on distinct Account ID

I think I know what is happening here. This is caused by the "Enable partitioned read" on the data source. DISTINCT and "Enable partitioned read" should not be used at the same time.
by jiegao
Wed Nov 16, 2011 2:28 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Join growth on distinct Account ID
Replies: 1
Views: 1481

Join growth on distinct Account ID

Hi, Currently we are doing a left join (hash partition on Account ID) on two Oracle tables on Account ID. The left table is a simple select statement with more than 1 unique Account ID, whereas the right statement has the 'Distinct' keyword and only selecting Account ID's from the table. My question...
by jiegao
Thu Nov 10, 2011 12:27 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Oracle db
Replies: 3
Views: 2228

What links are you mapping the output of the SQL query to?
by jiegao
Tue Nov 08, 2011 9:28 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to estimate sratchdisk space for sorting
Replies: 6
Views: 3284

Thanks. 50 Bytes is the max length of the row
by jiegao
Tue Nov 08, 2011 7:31 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to estimate sratchdisk space for sorting
Replies: 6
Views: 3284

Hi Ray, it looks like ways more than double of the entire volume of the data set. It requries 4 or 5 times more than the entire dataset. I need to justify the space requested. Thanks
by jiegao
Tue Nov 08, 2011 7:24 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to estimate sratchdisk space for sorting
Replies: 6
Views: 3284

PaulVL wrote:What is the total length of your row? Not just the keys.
The total length of the row is 50 bytes
by jiegao
Tue Nov 08, 2011 11:20 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to estimate sratchdisk space for sorting
Replies: 6
Views: 3284

How to estimate sratchdisk space for sorting

I have to sort data with volume of 900 million records. The sort is on two keys with 26 bytes in total. The sort is not stable sort. If I caluclate 900 million multiple by 26, it should be about 23 G of data. But it consumes more than 200G of scratch disk space. What does the other data come from? D...