Page 1 of 1

Dynamically handling file

Posted: Sat Mar 12, 2011 7:17 am
by san_dwh
Hi
I am new to datastage. In my project, one text file changing dynamically i.e, initially i have 25 fields.some times I am getting 30-35 fields. How can I handle this situation.

Thanks...

Re: Dynamically handling file

Posted: Sat Mar 12, 2011 12:03 pm
by soumya5891
If the no of fields in the source file get changed then the whole job flow will be affected.in this case what is the exact scenario?

Posted: Sat Mar 12, 2011 2:14 pm
by ray.wurlod
Are you reading or writing this file?

Posted: Mon Mar 14, 2011 8:27 am
by san_dwh
hi ray,

I am reading that file

Posted: Mon Mar 14, 2011 12:24 pm
by leomauer
Assumptions:
1. The file is delimited.
2. Only fields that exist in every file need to be processed and the varying part of the record is not processed.
3. The required fields are always in the same position in the record.

I would do it in transformer:
1. Define the input file record as unbound varchar (no length).
2. Use Field transformer function to parse the required field.

You can do something similar in non-delimited file by when assumption 2 and 3 still valid. In this case the Substring function instead of Field function.

But this kind of file sounds very unusual to me. Except for Cobol files with varying record length, I never encountered such file structure. I would question the source system first to verify if the file is produced consistently by the same program.

If the assumptions 2 and 3 are not true, I do not see the way to process the file.