File pattern

vij · Post by **vij** » Mon Mar 12, 2007 6:10 am

Hi,

I have some xml messages in a folder on UNIX server and I want these files to be read by DataStage.

1.Can I use "File Pattern to read these files? Is there any limitation in the number of files read using the File Pattern Technique?
2.If File pattern can be used, suggest me some other idea to read those files.

Note - all the files donot have the same metadata.

Thanks in advance!

chulett · Post by **chulett** » Mon Mar 12, 2007 6:40 am

'File Pattern' just means to use wildcards in the filename and, in essence, read more than one file in at a time. It has no bearing on the type of file, so of course you can use the feature with XML files. And there is no limitation on the number of files that I am aware of.

Your 'note' is another kettle of fish - how do you plan on processing them all in one job when they don't have the same metadata? Since these are allegedly XML files, will you somehow be trying to recognize what you've caught in your 'file pattern' net and use different XML Input stages and XPath Expressions to parse them accordingly?

ray.wurlod · Post by **ray.wurlod** » Mon Mar 12, 2007 7:54 am

A requirement of the File Pattern value is that all files share the same structure. Unless you're reading them as a single VarChar column, it appears that this is not the case in your situation.

kumar_s · Post by **kumar_s** » Mon Mar 12, 2007 9:36 pm

May the restriction as same as the number of argument that can be passed to "cat" command in OS. Not tested though.