XML Output- Aborting because of large number of records

[PRIYARAJ]

Hi,


When I write the data to Sequential file ,it runs without aborting. But when I try to write that to XML Output,the job aborts with error message no enough memory. The total record count is 1822963. So I want to split and write to XML output stage?CFfStage->Transformer->Xml Output(2). I think we can do it with limiting the row count to 1st(1-911481 records) & 2 nd(911482-1822963) file. I think we can do it with INROWNUM but I have not used that & don't know the sytax.

Can anyone help on how to proceed with this?

Thanks in Advance !!!

[chulett]

There's an option to use a Trigger Column in the XML Output stage, when the value in that column changes the stage closes the current file and opens a new output file.

[PRIYARAJ]

I see the Triggers in Transformer stage but I have no idea what to perform on that?

Can you please tell in detail what should I give there?

[chulett]

OK, edited my post to be more specific as to the stage I was referencing.

[PRIYARAJ]

I tried that but dint work because in there I have to select a column to trigger.Every field has different records & it is not pssible to do like that in my job.

Instead I tried @INROWNUM function in transformer stage.

I gave as constraint for first file as @INROWNUM < 400000 & for second link I gave as @INROWNUM >= 400000 .

Still,it is trying to insert more than 400000 records to first file. I am not sure where I am going wrong.

[ArjunK]

Try setting up a counter within the Transformer stage which changes its value i.e it gets incremented when the "Threshold" is reached. This Threshold can be set as a parameter that can be changed on need basis. Then use the Counter as the Trigger Column in the XML Output stage.

[chulett]

Exactly, this field does not need to be anything in your XML so can exist just for this purpose. Make sure it gets incremented properly, say once every 400000 records in you case.