File validation

[partha.natta]

I need to validate some source files with a single file which is having the details of the source file for e.g file name and record count of the source file and only the correct files will be processed for the next level.How can I implement the same in datastage windows environment.

[pandeesh]

From the source file, pass the file name as parameter to transformer , finally take the last record so that it contains the filename with total count.Then make a lookup with the file which contains filename and count.
Accordingly abort the job or proceed with next step

[Kryt0n]

So you are saying you have one file that identifies the source files that you should process? Does this one file contain only the files that should be processed? Or contain other file details too? A sequence to control this file list would be the easiest approach... there is a file pattern which can read a file list but I think that expects only file names not other details associated to the files...

[grimm336]

It seems you are getting a control file for the data file. if you have less info like data file name, record count and arrival time then i would suggest to do the data validation by using unix script. Also you need to check for the size of the data file. if it is huge or in GB you can create a temp table to dump the file and do the validation