DSXchange

Hi Nagin, I am new to datastage so I can not comment as to the best job type for your process. I can however gaurentee you that that above settings do work as I have just finished writing a few packages using the same method. I do however, agree with the comments from eostic. This could be a permiss...

Read the file as a single line (single VarChar) and use Field() function and constraint expressions in Transformer stage to parse records and filter by record type. ... Can anyone confirm if there are alternative methods for the job. I have a large file with header and trailer records and I would h...

Dear Ernie, Thanks for your reply. I was not entirely sure what you meant in your post being new to datastage. I have spoken to IBM and they have shared me the following solution which I think may be exactly what you were explaining. Step1: Create a job parameter of type pathname with the location o...

There is certainly no business need for them to load individually. The job will run overnight would not have a impact on the production system

Therefore, please continue with your educational exercise on the all files at once theory.

Paul

Thanks for your response.

There are 6 files roughly 300mb in size. Therefore I thought it may have been more performant to process them individually. Although being new to datastage I am not sure if these will be streamed or all 2gb loaded into memory.

Hi, I am new to datastage and I too have been suffering from the 512/3 character problem as above since following the tutorial at http://it.toolbox.com/wiki/index.php/Load_XML_files_using_a_DataStage_Parallel_job Following your advice I have replaced the sequential file stage with an External Source...