Processing multiple files

[chandra.shekhar@tcs.com]

Hi,

I have requirement where we need to process multiple files(say 100) on a daily basis. The mapping is a simple one-to-one i.e. 100 target tables for 100 files.
I face two challenges :-
a) All the files will have different metadata i.e different columns and will be loaded to different tables.
b) This has to be done using a single job/sequence.

Now this is new for me, I thought that we'll have 100 jobs but to do it in 1 job. Thats where I am stuck at.
Any suggestions.

[chulett]

Read your documentation on Runtime Column Propagation. It can be used with the Sequential File stage along with schema files to define the metadata.