Generate columns dynamically

[savis]

Hi,

Is there a way to generate columns dynamically?

If suppose, i have input files like:

file 1:

col1,col2,col3,col4

file 2:

col1,col2,col3,col4,col5

file 3:

col1,col2,col3,col4,col5,col6.

If the datatypes of all files are same, then can i use a single job for handling all the three input files?

please suggest..

Thanks,
Savis

[keshav0307]

Yes,
read teh record as a single column, and then in trasformer split the record into different columns, based on the delimiter position,
you can use field function for it

[kumar_s]

It depends on what you want to with the file. If you need to just read it and load it in output, then the above mentioned method can be used, or even schema file option can be availed.

[ag_ram]

As kumar_s stated, it depends on how far you have clear meaning of the word"handling" in your sentence

If the datatypes of all files are same, then can i use a single job for handling all the three input files?

[ray.wurlod]

The easiest complete approach is to use the Schema File property in the Sequential File stage, and set is value to a file that properly represents the record schema of the file that the generic job processes.