Hi
I am new to datastage. In my project, one text file changing dynamically i.e, initially i have 25 fields.some times I am getting 30-35 fields. How can I handle this situation.
Thanks...
Dynamically handling file
Moderators: chulett, rschirm, roy
-
soumya5891
- Participant
- Posts: 152
- Joined: Mon Mar 07, 2011 6:16 am
Re: Dynamically handling file
If the no of fields in the source file get changed then the whole job flow will be affected.in this case what is the exact scenario?
Soumya
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Assumptions:
1. The file is delimited.
2. Only fields that exist in every file need to be processed and the varying part of the record is not processed.
3. The required fields are always in the same position in the record.
I would do it in transformer:
1. Define the input file record as unbound varchar (no length).
2. Use Field transformer function to parse the required field.
You can do something similar in non-delimited file by when assumption 2 and 3 still valid. In this case the Substring function instead of Field function.
But this kind of file sounds very unusual to me. Except for Cobol files with varying record length, I never encountered such file structure. I would question the source system first to verify if the file is produced consistently by the same program.
If the assumptions 2 and 3 are not true, I do not see the way to process the file.
1. The file is delimited.
2. Only fields that exist in every file need to be processed and the varying part of the record is not processed.
3. The required fields are always in the same position in the record.
I would do it in transformer:
1. Define the input file record as unbound varchar (no length).
2. Use Field transformer function to parse the required field.
You can do something similar in non-delimited file by when assumption 2 and 3 still valid. In this case the Substring function instead of Field function.
But this kind of file sounds very unusual to me. Except for Cobol files with varying record length, I never encountered such file structure. I would question the source system first to verify if the file is produced consistently by the same program.
If the assumptions 2 and 3 are not true, I do not see the way to process the file.