Merge Stage Problem - deleting temporary file
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 385
- Joined: Tue Oct 07, 2003 4:55 am
Merge Stage Problem - deleting temporary file
Hi,
I have a simple job that merge 2 files into a sequential file (multiple instance). The merge process is OK in all invocations but the job always aborts because of :
mergeHoz.1.Merge_Hoz: Error occurred while deleting temporary file
I checked and there is no permissions problem (everybody has full control).
Does anyone has an idea ??
Amos
I have a simple job that merge 2 files into a sequential file (multiple instance). The merge process is OK in all invocations but the job always aborts because of :
mergeHoz.1.Merge_Hoz: Error occurred while deleting temporary file
I checked and there is no permissions problem (everybody has full control).
Does anyone has an idea ??
Amos
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
At a guess I'd say two instances of your job are trying to use the same temporary file name. I believe the merge stage creates a file behind the scenes that contains your merged data. It needs to remove this file when the job finishes. I don't have access to the eservices knowledge base however it could be a known defect.
Have you tried merging your files via an external script? In Unix you could merge two files with a before job command such as "cat file1 >> file2". This would run a lot faster than the merge stage but the files do need to be in the same format.
Have you tried merging your files via an external script? In Unix you could merge two files with a before job command such as "cat file1 >> file2". This would run a lot faster than the merge stage but the files do need to be in the same format.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
-
- Premium Member
- Posts: 385
- Joined: Tue Oct 07, 2003 4:55 am
If you end up wanting to further explore the Unix side of this answer, besides 'catting' the two files there are also the 'merge' and 'sort -u' that can be looked into. The 'merge' option should be very similar to what you are trying to accomplish in the job. Plus, 'sort unique' is another option if all you are trying to do is remove duplicate rows between the two files.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 385
- Joined: Tue Oct 07, 2003 4:55 am
Merge Stage Problem - deleting temporary file
Hi,
Make sure the both sequential files have records. I encountered the same problem if any one of the file does not have any records.
Hope this could help you.
Regards
Saravanan
Make sure the both sequential files have records. I encountered the same problem if any one of the file does not have any records.
Hope this could help you.
Regards
Saravanan
-
- Premium Member
- Posts: 385
- Joined: Tue Oct 07, 2003 4:55 am
Saravanan ,
Both my files exist and has records.
My job contained a merge stage --> transformer --> a shatred continer that does FTP. I changed the shared container to a local one - the temp folders stay after the job but I do not get the abort message anymore.
I'll report this to Ascential support next week.
Cheers,
Amos
Both my files exist and has records.
My job contained a merge stage --> transformer --> a shatred continer that does FTP. I changed the shared container to a local one - the temp folders stay after the job but I do not get the abort message anymore.
I'll report this to Ascential support next week.
Cheers,
Amos
-
- Premium Member
- Posts: 385
- Joined: Tue Oct 07, 2003 4:55 am
Hi,
I got no answer from Ascential support but I found an answer,
My job looked like this:
merge --> transformer --> seq file stage with sort filter command.
Ray was right - there was a timing problem, both the merge and the sort tried to use the same file. The funny thing is that even when I removed the sort command the job got corrupted and kept trying to do it ( i could see the sort process in the task manager and the output file was sorted).
The solution was to delete the job, recreate it and do the sort somewhere else.
Hope it helps you ,
Amos
I got no answer from Ascential support but I found an answer,
My job looked like this:
merge --> transformer --> seq file stage with sort filter command.
Ray was right - there was a timing problem, both the merge and the sort tried to use the same file. The funny thing is that even when I removed the sort command the job got corrupted and kept trying to do it ( i could see the sort process in the task manager and the output file was sorted).
The solution was to delete the job, recreate it and do the sort somewhere else.
Hope it helps you ,
Amos
Re: Merge Stage Problem - deleting temporary file
I agree with you Sarvanan.You are correct.This problem comes when any of the file has no records.I had faced the same issue.If both the files have records then this issue never comes up.
elavenil wrote:Hi,
Make sure the both sequential files have records. I encountered the same problem if any one of the file does not have any records.
Hope this could help you.
Regards
Saravanan