Page 1 of 1

How to capture the file name and row number

Posted: Thu Mar 08, 2007 10:42 pm
by syam
Hi All;

I am using the file pattern to read the source files.And during the rejection process i want to capture the row number and the file name into the error log table so that user can know which record is rejected and from which file in the file pattern is it rejected.
Thanks in advance

Thanks;
Syam.

Posted: Thu Mar 08, 2007 11:57 pm
by ray.wurlod
Not possible.

The reject link gets rows that do not match the defined metadata. You get the input row as a raw string, in one column. This is not negotiable.

There is a workaround, which is to read the file as a single VarChar column and parse it yourself in a Transformer stage. The Sequential File stage can add a File Name column. You could use a Filter command or stage variables to detemine the line number within the file. You may need to partition on File Name for the stage variable approach to work.

Posted: Fri Mar 09, 2007 1:48 am
by kumar_s
You can include RowNumberColumn in addition to FileNameColumn.

Posted: Fri Mar 09, 2007 7:42 am
by DSguru2B
Can't the business go off of a key column in the record? If not then add a running number to your file pre-hand before processing.

Posted: Fri Mar 09, 2007 8:13 am
by ray.wurlod
Hi Mister Hasty. If the OP is using File Pattern this implies sequential files and this implies no key.

Posted: Fri Mar 09, 2007 8:26 am
by DSguru2B
Agreed Ray ji. Hence, pre-process to attach a running number.

Posted: Fri Mar 09, 2007 8:31 am
by kumar_s
Wont it be simple to find the file name and row num directly using RowNumberColumn in addition to FileNameColumn rather than, Pre processing all the files and find the file names back with the help of running nubmer. :roll:

Posted: Fri Mar 09, 2007 8:54 am
by DSguru2B
Yes it would be. But as we are all in the mode of suggestions, I threw my poll ticket in as well.