Size limit on sequential file stage

bobbysridhar · Post by **bobbysridhar** » Wed Apr 04, 2012 3:16 pm

Hi,
I have the following requirement.
I have a lookup stage and in stage properties I put if look up fails then go to reject link. In the reject link I put sequential file.
The data is huge and I have to set size limit on sequentail file in reject link
as 2MB and once it reaches the 2MB limit it has to go to the new file.
So, I need to set up the size limit on sequential file and at the same time once the size limit reached it has to go automatically to another file.
My source and target both are oracle.
Could someone help me to find solution for this.
I need to put a timestamp in rejected records in sequential file.
I can create timestamp in transformer, but do not know how to pass it to sequential reject file through lookup failure link.

Thanks,
Sridhar

ray.wurlod · Post by **ray.wurlod** » Wed Apr 04, 2012 3:28 pm

Welcome aboard.

You can't do that with DataStage.

You might try creating the large file then using a UNIX command like split to break it up. This will work even though you are executing on a Windows server - every DataStage installation has MKS Toolkit (UNIX emulator).

qt_ky · Post by **qt_ky** » Wed Apr 04, 2012 5:12 pm

2 MB is rather small. Are you sure the limit should not be 2 GB instead?

If you're having trouble adding a timestamp on the reject link, you could insert a Transformer stage before the Lookup stage, and derive the timestamp there. If the row is rejected, then it already has the timestamp. If it's not rejected, you can drop the column from normal Lookup stage output.

kandyshandy · Post by **kandyshandy** » Thu Apr 05, 2012 3:22 am

It's going to be a public holiday tomorrow... but some questions like this make me think like going home instead of fun filled places