DSXchange

I have a job sequence which has an execute command stage looking for a specific file pattern, and when the file is found another job is invoked to load the file into the database after applying transformations.

So far, files have been coming in from two different sources with same metadata. Now, as the requirement changed one of the source files metadata slightly changed. Now, its bringing in couple of columns more than earlier. But file name pattern is same as earlier.

So, what I am trying to do is, read the file on the server and depending on how many columns it has or depending on the kind of data it has i want to direct it to different stages.

Now, if i chose Build stage rather than doing this through a shell script, would I be able to read the file in to the Build Stage with out specifying the metadata?

I appreciate your thoughts/input on this.

Thanks.

Yes, but you could do that with regular stage types as well. For example you could read the file with each line being a single VarChar column, and parse it within the job, possibly with an appropriate Column Import stage, possibly with a Transformer stage.

Or, particularly if there are not too many different formats, you could create copies of the job and have one for each fully-defined metadata format. That's the way I'd prefer to go.