Merge Stage Issue: Joining two sequential files

kgfrancis · Post by **kgfrancis** » Thu Feb 03, 2005 9:51 pm

I am new to DataStage server, and the issue that I am encountering is that when I use the merge stage to perform a full outer join using the complete set command. I get the following error:

Abnormal termination of stage testMergeA. Sequential_File_5.IDENT1 detected

I want to match like records in both sequential files and return one record in the resulting output sequential file. Also, this output sequential file should return all records that did not find a match.

I have seen another post that suggest appending the two files, but this will create duplicate records which is not ideal for my situation.

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Fri Feb 04, 2005 2:54 am

Did you sort the data before sending through to merge?

PhilHibbs · Post by **PhilHibbs** » Fri Feb 04, 2005 5:01 am

Sainath.Srinivasan wrote:Did you sort the data before sending through to merge?

Is this necessary? I thought the merge stage worked by building up hash files. I have merged unsorted data, including a many-to-many join and a full outer join (although I think the latter files did happen to be sorted anyway).

Did you set a temporary directory, kgfrancis?

rmh3 · Post by **rmh3** » Mon Feb 07, 2005 2:41 pm

I'm having issues with the Merge stage as well. I received the Abnormal Termination message the first time, but now I'm seeing the one below. Unfortunately, I tried so many different things, I'm not sure which one switched the error message at this point.

tempDirStageStuDemogCateService2002up..soCATEStuDemogAB.IDENT1: Pin 1 not initialized

I've never tried to use the merge stage before so I may be missing something fundemental here. Incidentally, the "View Data..." button stays grayed out as well inside of Designer.

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Mon Feb 07, 2005 5:34 pm

You can try to increase the tmp dir size of point to a larger space by defining a new path in the uvconfig file. You will need to regen DataStage kernel to bring this to effect.

PhilHibbs · Post by **PhilHibbs** » Tue Feb 08, 2005 4:33 am

rmh3 wrote:Incidentally, the "View Data..." button stays grayed out as well inside of Designer.

Mine too, I think that's normal. I think it's a redundant button that's only there because of the window template that that component uses.

roy · Post by **roy** » Tue Feb 08, 2005 9:13 am

Hi,
In my expiriance server merge stage is not a good idea for merging large files, your probably better off with hash file lookups.
I once tried to merge 2 big files (2+ digit GB each) on true 64 and it was a no go, on the other hand it went nice with small files.

also some times it seems the only way to make it work after a change is to rebuild the entire stage

.

IHTH,