Page 1 of 1

File validation

Posted: Thu Feb 16, 2012 1:42 am
by partha.natta
I need to validate some source files with a single file which is having the details of the source file for e.g file name and record count of the source file and only the correct files will be processed for the next level.How can I implement the same in datastage windows environment.

Posted: Thu Feb 16, 2012 2:00 am
by pandeesh
From the source file, pass the file name as parameter to transformer , finally take the last record so that it contains the filename with total count.Then make a lookup with the file which contains filename and count.
Accordingly abort the job or proceed with next step

Posted: Thu Feb 16, 2012 3:17 pm
by Kryt0n
So you are saying you have one file that identifies the source files that you should process? Does this one file contain only the files that should be processed? Or contain other file details too? A sequence to control this file list would be the easiest approach... there is a file pattern which can read a file list but I think that expects only file names not other details associated to the files...

Re: File validation

Posted: Wed Feb 22, 2012 10:53 am
by grimm336
It seems you are getting a control file for the data file. if you have less info like data file name, record count and arrival time then i would suggest to do the data validation by using unix script. Also you need to check for the size of the data file. if it is huge or in GB you can create a temp table to dump the file and do the validation