fatal error

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
pavan_test
Premium Member
Premium Member
Posts: 263
Joined: Fri Sep 23, 2005 6:49 am

fatal error

Post by pavan_test »

hello.,

<GridFiles,0> Source subproc: xargs: environment is too large for exec
cat: write error: Broken pipe
<GridFiles,0> Filter status 256
Filter process failed: 1
Import error at record 0


can anyone please tell me what is this error is and how to fix it.

thanks
pan
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

the error you are getting is coming from the UNIX command xargs. The command that you are generating is bigger than what your environment can handle.

What are you doing in the stage "GridFiles"?
pavan_test
Premium Member
Premium Member
Posts: 263
Joined: Fri Sep 23, 2005 6:49 am

fatal error

Post by pavan_test »

[quote="ArndW"]the error you are getting is coming from the UNIX command [b]xargs[/b]. The command that you are generating is bigger than what your environment can handle.

What are you doing in the stage "GridFiles"?[/quote]

hello.,


i am using the external filter stage, in the output tab -

source program = cat #Source_Dir_Pre#/#PreviousFileList# | xargs zcat -f

where previousFileList = file.txt

file.txt has .,

/home/sk/filename1
/home/kk/filename2
/home/pk/filename3

the external filter stage will take each file and has to concatenate the data in each file
and output it.

thanks
pan
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

And your command is running correctly when you manually test it from UNIX with the appropriate values? Might you use the xargs -s option to get this to work? It doesn't seem to be a DataStage issue here, just a UNIX one.
pavan_test
Premium Member
Premium Member
Posts: 263
Joined: Fri Sep 23, 2005 6:49 am

fatal error

Post by pavan_test »

[quote="ArndW"]And your command is running correctly when you manually test it from UNIX with the appropriate values? Might you use the xargs -s option to get this to work? It doesn't seem to be a DataStage issue here, just a UNIX one.[/quote]

hello.,

can you please tell me what does the xargs -s do..?

thanks
pan
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Pavan,

the -s <size> lets you specify the size. The exact description can be retrieved by checking the man pages for xargs. I think this does differ from environment to environment and also on different platforms.
Post Reply