file column name mixed up when files number is greater nodes

mdan · Post by **mdan** » Wed Nov 07, 2007 5:23 am

Hi,
I wonder if anyone got this strange behavior:
I'm reading the files using the File Pattern as Read Method and File Name column in order to get the file name. It reads all the files correctly (about 500 in the original job - 5 in this example), but in the file name column I got only one file name per node (the testing configuration has 4 nodes - looks like I'm getting the last file name per each node).

Code: Select all


simple job
SeqStage -> Copy -> SeqFile

e.g.
files TEST01 to TEST05
each file content is related to the file name (ONE -> TEST01, ...)

result
TWO,/PATH_TO_FILE/TEST02
FOUR,/PATH_TO_FILE/TEST04
THREE,/PATH_TO_FILE/TEST03
ONE,/PATH_TO_FILE/TEST05
FIVE,/PATH_TO_FILE/TEST05

Thanks,
Dan

balajisr · Post by **balajisr** » Wed Nov 07, 2007 6:02 am

Can you post the code entered in "File Pattern" property?

JoshGeorge · Post by **JoshGeorge** » Wed Nov 07, 2007 7:03 am

Set APT_IMPORT_PATTERN_USES_FILESET to True and try. If this didn't work you might have to get a patch from IBM for fixing this issue. Been discussed this before here.

mdan · Post by **mdan** » Wed Nov 07, 2007 7:49 am

JoshGeorge wrote:Set APT_IMPORT_PATTERN_USES_FILESET to True and try. If this didn't work you might have to get a patch from IBM for fixing this issue. Been discussed this before here.

Thanks,
I've seen the post. Looks like I have to go for that patch ...

regards and thanks,
Dan