Search found 71 matches
- Sun Jul 15, 2007 6:22 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: uvodbc.config
- Replies: 5
- Views: 1758
You can specify the service/database details in .odbc.ini. Please refer to the online documentation for details :) Interesting... I''ve always used 'Case 2' without issue regardless of database server location. Thanks Roy and Chulett for your information. Even in case 2, what is the Service ? Is it ...
- Thu Jun 14, 2007 8:54 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: (Server=Dedicated) parameter in TNSNAMES.ORA file
- Replies: 4
- Views: 6453
Re: (Server=Dedicated) parameter in TNSNAMES.ORA file
In my opinion, (SERVER = DEDICATED) should be there in the TNSNAMES.ORA if the database is set up with DEDICATED Server option. It doesn't affect anything to add that parameter into the TNSNAMES.ORA if it's not there. Hi All, We have a condition here in Data Stage 7.5.2 with server jobs. Currently, ...
- Tue Mar 13, 2007 6:37 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
:cry: I wish there was a perfect hashing algorithm to guarantee uniqueness. That is an issue then. Thanks, Ray. There's one major drawback. It does not guarantee uniqueness. For example CRC32 has about a 1 in 4000000 chance of generating a duplicate. Four million is not that many rows. The entire ra...
- Tue Mar 13, 2007 6:03 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
Hi Craig, thanks for the reply. I am still looking for the suitable function and haven't decided to use CRC32 or MD5 yet. I know the traditional surrogate key is a good choice but just want to get some comments on what will be the drawback of the hashing approach. I know no-one has asked for it, but...
- Tue Mar 13, 2007 5:59 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
Craig, I am still looking for that suitable hashing algorithm and haven't decided to use CRC32 or MD5. I know the traditional surrogate key is a good choice, but just want to know what is the drawback of the hashing approach? Thank you. I know no-one has asked for it, but thought I would throw in my...
- Tue Mar 13, 2007 5:54 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
Hi DSguru, why do you think running numbers is better choice of SKs? I know typical hash functions will return a fixed-length string. I haven't decided which function to use, and I am not sure if hash function could return numbers.
DSguru2B wrote:Likewise Craig. Stick to generation of running number.
- Tue Mar 13, 2007 4:28 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: HOW TO RESTART THE JOB
- Replies: 3
- Views: 1590
Re: HOW TO RESTART THE JOB
Well, it depends on your requirement and preference. Some of my options will be: 1. truncate and reload the whole 2. Replace/overwrite 3. Take the whole load as a single transaction 4. Do a lookup and only load the rest of records when reloading 5. More... Hi How can I restart the job, if job get ab...
- Tue Mar 13, 2007 3:28 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
Hi Kumar_s, to give you more details, I have a simplified job below: sequencial file =>Transformer=>some additional logic 1=>load into target database (DB 1) ................................|| ................................\/ ..........................Some addtional logic 2 ..........................
- Tue Mar 13, 2007 1:12 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
Good point. Thanks. It should be ok as long as the records in DB1 get SKs allocated. I am assuming that records will be loaded into DB2 (the data quality database) without failure, if they do fail, they have to be fixed and loaded into DB2 again until success...that means, all exceptions will be loa...
- Tue Mar 13, 2007 12:24 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
That's the magic of hash functions. If I apply the same function in two places (one in the ETL flow of loading DB1 and the other in the ETL flow of DB2), I would expect I get the same SKs for the same NKs for both. SK is just an additional key apart form the natural key which are created for some pu...
- Mon Mar 12, 2007 11:50 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
- Mon Mar 12, 2007 10:45 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Generating Surrogate Keys via hash functions
- Replies: 20
- Views: 8020
Generating Surrogate Keys via hash functions
My job is to load flat files into the target database (DB1 for short). Some records will have data quality issues, e.g. where a mandatory field has no vaule, but will be corrected and still get to load into the database. At the same time these records are required to be captured and loaded into a se...
- Thu Mar 08, 2007 4:55 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Transformer logic
- Replies: 6
- Views: 3723
Re: Transformer logic
The function IsValid can be used to check if a value is numeric, I am using Transformer stage variable to keep the result and then it can be used in the constraint condition. Hi Guys I have to do a some sort of validation process in the datastage transformer. The logic to be used is=> If the input v...
- Thu Mar 08, 2007 4:51 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: type conversion
- Replies: 8
- Views: 2302
Re: type conversion
That means StringToTimestamp fails. Check the format of the input value where you got the output **********. Hi- I have one funny thing going on with my job. One of the fields in my job needs string to timestamp conversion for that I used the fallowing function. StringToTimestamp((DSLink5.SrcFileDt)...
- Thu Mar 01, 2007 4:56 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Reading a Sequential File
- Replies: 14
- Views: 5513
You will need to set up an additional property for variable-length fields for fixed-width files. Double click each column in the Column page of the sequencial file stage and in the popup window Edit Column Meta Data set the Field Width property for each of the fields. It should work for all data typ...