Page 1 of 1

file column name mixed up when files number is greater nodes

Posted: Wed Nov 07, 2007 5:23 am
by mdan
Hi,
I wonder if anyone got this strange behavior:
I'm reading the files using the File Pattern as Read Method and File Name column in order to get the file name. It reads all the files correctly (about 500 in the original job - 5 in this example), but in the file name column I got only one file name per node (the testing configuration has 4 nodes - looks like I'm getting the last file name per each node).

Code: Select all


simple job
SeqStage -> Copy -> SeqFile

e.g.
files TEST01 to TEST05
each file content is related to the file name (ONE -> TEST01, ...)

result
TWO,/PATH_TO_FILE/TEST02
FOUR,/PATH_TO_FILE/TEST04
THREE,/PATH_TO_FILE/TEST03
ONE,/PATH_TO_FILE/TEST05
FIVE,/PATH_TO_FILE/TEST05
Thanks,
Dan

Posted: Wed Nov 07, 2007 6:02 am
by balajisr
Can you post the code entered in "File Pattern" property?

Posted: Wed Nov 07, 2007 7:03 am
by JoshGeorge
Set APT_IMPORT_PATTERN_USES_FILESET to True and try. If this didn't work you might have to get a patch from IBM for fixing this issue. Been discussed this before here.

Posted: Wed Nov 07, 2007 7:49 am
by mdan
JoshGeorge wrote:Set APT_IMPORT_PATTERN_USES_FILESET to True and try. If this didn't work you might have to get a patch from IBM for fixing this issue. Been discussed this before here.
Thanks,
I've seen the post. Looks like I have to go for that patch ...

regards and thanks,
Dan