Search found 114 matches

by clshore
Sat Dec 10, 2005 2:37 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Maximum stages in a job
Replies: 15
Views: 7203

I was on an EE assignment where the number of stages exceeded 1,000 in several jobs. My firm was called in midway through the project, after the jobs were written, to help resolve 'some issues'. After some modifications, and much tweaking of kernel, memory, and disk resources, the jobs did actually ...
by clshore
Mon Nov 28, 2005 4:21 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Reserved Character Conversion
Replies: 12
Views: 5021

Generally, the tr command will yield the best performance when translating character values on a UNIX system.

Carter
by clshore
Wed Oct 26, 2005 12:24 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to compile an enternal C/C++ Routine and use in DS PX
Replies: 11
Views: 5446

Another caveat, my documentation (7.5) specified that a C++ compiler was required.
I have not tried (yet) on a Windows platform.

Carter
by clshore
Thu Oct 20, 2005 4:21 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: 10 pounds in a 5 pound bag
Replies: 28
Views: 6770

OK, 200 input characters (assume a-z,A-Z,0-9, about 64 possibilites?). Each segment is 40 of these characters, so the number of different possible character patterns is: 64^40 = 1.76 x 10^72 Each of these segments is translated by the CRC function into an 8 character string of hex. The number of dif...
by clshore
Thu Oct 20, 2005 12:24 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: File Writing Limitations
Replies: 2
Views: 1443

Are you writing a routine just so you can append?
Otherwise, use a Sequential file stage, and click 'Append to existing file' on the Inputs tab.

Carter
by clshore
Thu Oct 20, 2005 11:31 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Time and TimeStamp with microseconds
Replies: 20
Views: 13687

I needed my microseconds now, so I added an external routine for the purpose: /* getSysUsec.c gets sys seconds and usecs, formats as "%d.%06d", returns string */ #include <unistd.h> #include <stdio.h> #include <time.h> #include <sys/time.h> char * getSysUsec(){ struct timeval time; char re...
by clshore
Thu Oct 20, 2005 11:12 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to skip the trailer record
Replies: 25
Views: 12905

Hmm..., that's exactly the point. You said that you were using the CFF stage, right? Have you read the functional description of what a CFF does? In COBOL, the REDEFINES clause allows different layout structures for records contained in the same file. See if you can obtain the COBOL FD for the file ...
by clshore
Wed Oct 19, 2005 12:11 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to skip the trailer record
Replies: 25
Views: 12905

You are already using CFF, so what did you use for file metadata? EBCDIC files with headers/trailers typically are defined in terms of a COBOL file structure, where the header and trailer are REDEFINES of the base record structure. I believe that the CFF will handle such correctly, as long as the sc...
by clshore
Wed Oct 19, 2005 12:44 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Sequential File stage need to select latest File to Extract
Replies: 9
Views: 6670

You can use the ls -1t method, but some folks have the unfortunate habit when viewing files, of writing a new version (shift ZZ, you know who you are), which resets the UNIX file timestamp. You will be lucky to find out about one of these; more often the wrong file is silently processed, the details...
by clshore
Tue Oct 18, 2005 6:42 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Time and TimeStamp with microseconds
Replies: 20
Views: 13687

Arnd,
Did you ever get the microseconds to show?

I'd much rather use a DataStage built-in to get usec, than roll custom code with gettimeofday() etc.

Carter
by clshore
Thu Oct 13, 2005 8:25 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to Omit last record from a sequential file
Replies: 13
Views: 11282

It seems from your example that the second column of Trailer
records always contains 'TRA'?
So, since you are using a Transformer, why not just add a constraint like this:

col2 <> "TRA"

Carter
by clshore
Tue Oct 11, 2005 5:47 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Julian date conversion to yyyy-mm-dd : hh-mm-ss
Replies: 19
Views: 11834

Be careful with your assumption that all dates will be in 21 century. The code below will fail with an input date of 99032 (1999-02-01). The lower 3 digits are the day of the year, the remaining upper digits are years since 1900. Carter Hello folks, After 3 days of work with all the iconv,oconv and ...
by clshore
Mon Oct 10, 2005 7:59 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Julian date conversion to yyyy-mm-dd : hh-mm-ss
Replies: 19
Views: 11834

I believe you must add 1900 to the first result, not 1899:

ex) 105030 = 2005-01-30
Int(105030/1000) = 105
105 + 1900 = 2005

Carter
by clshore
Thu Oct 06, 2005 5:51 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Julian to Gregorian(DB2 format YYYYMMDD) date conversion
Replies: 6
Views: 8196

For dates in years between 1900 and 9999, you can convert in DB2 like this: DB2 internal date = DATE(RTRIM(CHAR( D1 + 1900000 ))) where D1 is the julian-like-date Using your example of 105030, DATE(RTRIM(CHAR( 105030 + 1900000 ))) DATE(RTRIM(CHAR( 2005030 ))) DATE(RTRIM('2005030 ')) -- DB2 creates 1...
by clshore
Thu Oct 06, 2005 3:22 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Changing file Permission through shell script by Ftp process
Replies: 14
Views: 6346

I have seen some ftp that suppport umask for the target files. But most ftp implementations support some kind of local shell capability, so you can say something like this in your ftp script after the files have been recieved. .... get $filename .... ! chmod 111 $filename .... Check the documentatio...