Page 1 of 1

Search & get a value from a file in Datastage

Posted: Thu Dec 19, 2013 5:33 am
by Arpanakhade2009
I have a reference file with the details of the fields as below:

field name, default value, length
col1, US, 10
col2, UK, 35
col3,, 10

I have the input file as;
col1, col2, col3
US, UK, NNNNNNNNNNNN
US, uk, BBBBB
US,UK,AAAA

The first 2 records will be sent to error as the length for col3 does not match with reference value and for 2nd record the default value does not match.

Is it possible to get the column name in transformer or will i need to hard code 'col1' in transformer to use as key & perform lookup on reference and get the details.

Read in an article that we cant use server routine in parallel jobs. For parallel routine i am unable to know the g++ version.

I am new to routines, can you please help me to write routines to get the field details.

Please suggest if an alternate way to get the desired output.

Posted: Thu Dec 19, 2013 7:36 am
by ArndW
I am not sure I understand your exact rules, but you won't need to write a routine for this. You do need to hard-code the reference column in lookups.

Posted: Thu Dec 19, 2013 10:29 pm
by Arpanakhade2009
The rule is as below:
For Column 1, the default value should always be 'US' but i have 100 jobs with this logic and around 100 fields.
If i want to change the default value tomorrow, I will need to modify in 100 jobs. Instead, I can just modify a reference file, which has the default value and my job is done.

Same goes for length check
Thanks.

Please advise a logic.

Posted: Fri Dec 20, 2013 5:58 am
by ArndW
Ah, now I see what you are looking for, a dynamic method using one file to define the metadata for reading another.
Have you looked into schema files? You could use one job to read your reference file and create a schema file, which is then used to read the data file.

Posted: Fri Dec 20, 2013 6:03 am
by Arpanakhade2009
Can you please provide stage level logic to do the same, cannot get what you are trying to say.

Posted: Fri Dec 20, 2013 6:17 am
by ArndW
Have you looked into schema files and how they can be used?