Search & get a value from a file in Datastage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Arpanakhade2009
Participant
Posts: 30
Joined: Tue Sep 08, 2009 4:24 am

Search & get a value from a file in Datastage

Post by Arpanakhade2009 »

I have a reference file with the details of the fields as below:

field name, default value, length
col1, US, 10
col2, UK, 35
col3,, 10

I have the input file as;
col1, col2, col3
US, UK, NNNNNNNNNNNN
US, uk, BBBBB
US,UK,AAAA

The first 2 records will be sent to error as the length for col3 does not match with reference value and for 2nd record the default value does not match.

Is it possible to get the column name in transformer or will i need to hard code 'col1' in transformer to use as key & perform lookup on reference and get the details.

Read in an article that we cant use server routine in parallel jobs. For parallel routine i am unable to know the g++ version.

I am new to routines, can you please help me to write routines to get the field details.

Please suggest if an alternate way to get the desired output.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I am not sure I understand your exact rules, but you won't need to write a routine for this. You do need to hard-code the reference column in lookups.
Arpanakhade2009
Participant
Posts: 30
Joined: Tue Sep 08, 2009 4:24 am

Post by Arpanakhade2009 »

The rule is as below:
For Column 1, the default value should always be 'US' but i have 100 jobs with this logic and around 100 fields.
If i want to change the default value tomorrow, I will need to modify in 100 jobs. Instead, I can just modify a reference file, which has the default value and my job is done.

Same goes for length check
Thanks.

Please advise a logic.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Ah, now I see what you are looking for, a dynamic method using one file to define the metadata for reading another.
Have you looked into schema files? You could use one job to read your reference file and create a schema file, which is then used to read the data file.
Arpanakhade2009
Participant
Posts: 30
Joined: Tue Sep 08, 2009 4:24 am

Post by Arpanakhade2009 »

Can you please provide stage level logic to do the same, cannot get what you are trying to say.
Arpana
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Have you looked into schema files and how they can be used?
Post Reply