Merge Stage Problem - deleting temporary file

Amos.Rosmarin · Post by **Amos.Rosmarin** » Tue Oct 14, 2003 6:15 am

Hi,

I have a simple job that merge 2 files into a sequential file (multiple instance). The merge process is OK in all invocations but the job always aborts because of :

mergeHoz.1.Merge_Hoz: Error occurred while deleting temporary file

I checked and there is no permissions problem (everybody has full control).

Does anyone has an idea ??

Amos

vmcburney · Post by **vmcburney** » Tue Oct 14, 2003 11:49 pm

At a guess I'd say two instances of your job are trying to use the same temporary file name. I believe the merge stage creates a file behind the scenes that contains your merged data. It needs to remove this file when the job finishes. I don't have access to the eservices knowledge base however it could be a known defect.

Have you tried merging your files via an external script? In Unix you could merge two files with a before job command such as "cat file1 >> file2". This would run a lot faster than the merge stage but the files do need to be in the same format.

Amos.Rosmarin · Post by **Amos.Rosmarin** » Wed Oct 15, 2003 2:03 am

Vincent ,

Each instances creates it's own temp directory so I think there is no conflict, nevertheless even when I run it as a single instances job the same error happens.

I do not need to concatenate files, I need a real join based on common keys. The merge stage is the best way to do.

Amos

chulett · Post by **chulett** » Wed Oct 15, 2003 7:02 am

If you end up wanting to further explore the Unix side of this answer, besides 'catting' the two files there are also the 'merge' and 'sort -u' that can be looked into. The 'merge' option should be very similar to what you are trying to accomplish in the job. Plus, 'sort unique' is another option if all you are trying to do is remove duplicate rows between the two files.

Amos.Rosmarin · Post by **Amos.Rosmarin** » Wed Oct 15, 2003 7:23 am

I want to know if anyone had seen this problem of 'Error occurred while deleting temporary file ' in a merge or sort stages ??

The 'pure inner join' i'm looking for can not be done by command line and anyway I do not have unix.

chulett · Post by **chulett** » Wed Oct 15, 2003 7:43 am

Doesn't seem like it so far.

Have you opened up a ticket with support? Could be something specific with the 7.0 release...

elavenil · Post by **elavenil** » Sat Oct 18, 2003 1:09 am

Hi,

Make sure the both sequential files have records. I encountered the same problem if any one of the file does not have any records.

Hope this could help you.

Regards
Saravanan

Amos.Rosmarin · Post by **Amos.Rosmarin** » Sat Oct 18, 2003 8:27 am

Saravanan ,

Both my files exist and has records.

My job contained a merge stage --> transformer --> a shatred continer that does FTP. I changed the shared container to a local one - the temp folders stay after the job but I do not get the abort message anymore.

I'll report this to Ascential support next week.

Cheers,
Amos

raviyn · Post by **raviyn** » Wed Nov 19, 2003 10:26 pm

Did you get any response from Ascential about it. I am on DS v6.0.1 and I am also getting the same error.

Amos.Rosmarin · Post by **Amos.Rosmarin** » Thu Nov 20, 2003 2:23 am

Hi,

I got no answer from Ascential support but I found an answer,

My job looked like this:

merge --> transformer --> seq file stage with sort filter command.

Ray was right - there was a timing problem, both the merge and the sort tried to use the same file. The funny thing is that even when I removed the sort command the job got corrupted and kept trying to do it ( i could see the sort process in the task manager and the output file was sorted).

The solution was to delete the job, recreate it and do the sort somewhere else.

Hope it helps you ,
Amos

bapajju · Post by **bapajju** » Thu Nov 20, 2003 9:13 pm

I agree with you Sarvanan.You are correct.This problem comes when any of the file has no records.I had faced the same issue.If both the files have records then this issue never comes up.

elavenil wrote:Hi,

Make sure the both sequential files have records. I encountered the same problem if any one of the file does not have any records.

Hope this could help you.

Regards
Saravanan

DSXchange