Hello everyone,
i'm not an beginner in designing DS job, but i tried to create and append text file, but it seems that the "append" action overwrote the current first rows (from top to bottom). It's very annoying. I have to precise that there is only one source (text file), one transformer, and from this transformer, there are two seq. file stage, the first to create the file, and the second to append "transformed" rows and i set the correct link order (1- create 2-append). But unfortunately, the "append" overwrote from the top and not from the bottom of the text file. What is wrong ? Is it a bug ? i was thinking that the "append" action always write from the end of the file.
Thanks for your answers
Chanthou
create and append text to sequential file in the same stage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 11
- Joined: Mon Jun 16, 2003 2:21 am
This seems to be a popular topic lately.
For one example, please see this thread in the EE/PX forum.
You cannot have two processes writing to the same sequential file at the same time. Period. If your create link is writing rows to the file, it must complete and close the file before it is safe for the append action to begin. It's not a bug or a DataStage problem, it is how sequential files work.
![Wink :wink:](./images/smilies/icon_wink.gif)
You cannot have two processes writing to the same sequential file at the same time. Period. If your create link is writing rows to the file, it must complete and close the file before it is safe for the append action to begin. It's not a bug or a DataStage problem, it is how sequential files work.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Participant
- Posts: 12
- Joined: Tue Dec 21, 2004 7:57 am
The output to the sequential file is in the same order as the input. So if the rows first satisfy the 'append' constraint, then those rows are written first. Specifying the link order in sequential file stages doesn't ensure that all outputs to the 'create' link are written first followed by the 'append' link.
Wilcy William
-
- Participant
- Posts: 11
- Joined: Mon Jun 16, 2003 2:21 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
That's what you had to do.
The "create" stage has not closed the file, and chances are that it had not actually created the file (rows had been buffered in memory) before the "append" stage was activated and therefore determined that it, too, must write to a new, empty file.
With sequential files, there can only be one writer at a time, so the updates through the earlier stage were overwritten.
Your separated solution is the correct one.
The "create" stage has not closed the file, and chances are that it had not actually created the file (rows had been buffered in memory) before the "append" stage was activated and therefore determined that it, too, must write to a new, empty file.
With sequential files, there can only be one writer at a time, so the updates through the earlier stage were overwritten.
Your separated solution is the correct one.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.