ds_ipcput() - row too big for inter stage rowbuffer

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
FDW_CITI
Participant
Posts: 13
Joined: Sun Aug 31, 2008 10:18 pm

ds_ipcput() - row too big for inter stage rowbuffer

Post by FDW_CITI »

Hi,

I have created a server shared container for reading XML file. It contains the stages as follows:

Folder stage -- > Transformer --> XML Input stage.

I am using this shared container in my parallel job where I am getting the error "ds_ipcput() - row too big for inter stage rowbuffer".

Can anyone advice on this error.

Regards,
Manju
gssr
Participant
Posts: 243
Joined: Fri Jan 09, 2009 12:51 am
Location: India

Post by gssr »

Search the forum before posting,

Hope this will help....

viewtopic.php?t=132107&highlight=rowbuffer
RAJ
FDW_CITI
Participant
Posts: 13
Joined: Sun Aug 31, 2008 10:18 pm

Post by FDW_CITI »

Hi,

I have already gone through the post.

When I am trying to read the XML file (1600KB) in Server, it is working fine.

The same refered in Parallel job (v8.1) as shared container is giving the error "ds_ipcput() - row too big for inter stage rowbuffer".

I tried turning off the row buffer. Even then the same issue.

Also tried setting the below Parameter:

$APT_MAX_TRANSPORT_BLOCK_SIZE=1048576
$APT_AUTO_TRANSPORT_BLOCK_SIZE=False
$APT_DEFAULT_TRANSPORT_BLOCK_SIZE=1048576

But still getting the same error.

Regards,
Manju


gssr wrote:Search the forum before posting,

Hope this will help....

viewtopic.php?t=132107&highlight=rowbuffer
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Your block is 1.6Mb and you set the maximum block size to 1Mb and are surprised that the block doesn't fit?
FDW_CITI
Participant
Posts: 13
Joined: Sun Aug 31, 2008 10:18 pm

Post by FDW_CITI »

Hi,

I am able to solve this issue by removing the Transformer stage in my Shared Container. I had used Transfomer stage to filter out a single file name fetched using the Folder stage. Now, I am able to read a single file using the Folder stage itself by passing the file name in wildcard property in this stage.

Thanks.

Regards,
Manju
Post Reply