Hi ,
Is it possible to write to a same sequential file from two transformer stages in a job ? The metadata written by both the transformer stage is same.I tried doing this by giving overwriting existing file in one link & Append to existing file in other seq file link.
writing to a same sequential file from two transformer stage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 90
- Joined: Mon Dec 08, 2003 4:48 am
- Location: Chennai
writing to a same sequential file from two transformer stage
Think Ahead,
Raj.D
Raj.D
Hi,
writing to a sequential file in parallel is not supported and might cause unreliable data!
if you want simply to write to a sequential file twicw in the same job, again not in parallel, you'r better off writing 2 seperate stages each writing at it's time, this will probably reduce the chance of future changes to the job won't cause parallel writing ot htis file (I'd add an anotation specifying this in the job as well).
IHTH
writing to a sequential file in parallel is not supported and might cause unreliable data!
if you want simply to write to a sequential file twicw in the same job, again not in parallel, you'r better off writing 2 seperate stages each writing at it's time, this will probably reduce the chance of future changes to the job won't cause parallel writing ot htis file (I'd add an anotation specifying this in the job as well).
IHTH
Roy R.
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
![Image](http://www.worldcommunitygrid.org/images/logo.gif)
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
![Image](http://www.worldcommunitygrid.org/images/logo.gif)
-
- Participant
- Posts: 90
- Joined: Mon Dec 08, 2003 4:48 am
- Location: Chennai
sdevashis wrote:I was experimenting now and found that it goes in a sequential mode. So I am wondering in what scenarios, the data can be unreliable.
I guess unless for constrainsts, which sequential files are stupid at....rest everything should go fine.
HEY HOW DID YOU DO THAT? I GOT AN ERROR WHEN I TRIED DOING THAT WITH FIXED WIDTH FILES.
Think Ahead,
Raj.D
Raj.D
Hi,
If some writing goes from both links at the same time to the file, data may be unreliable.
a sequential file stage is not ment to have parallel writing performed on it.
if you can be sure this won't happen, do it at your own risk.
I would prefer to seperate the 2, you could build 2 seperate files with the same definition and concatenate them when you'r done to form the final file you need (whethere be it via pure DS or an OS command run from DS)
IHTH
If some writing goes from both links at the same time to the file, data may be unreliable.
a sequential file stage is not ment to have parallel writing performed on it.
if you can be sure this won't happen, do it at your own risk.
I would prefer to seperate the 2, you could build 2 seperate files with the same definition and concatenate them when you'r done to form the final file you need (whethere be it via pure DS or an OS command run from DS)
IHTH
Roy R.
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
![Image](http://www.worldcommunitygrid.org/images/logo.gif)
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
![Image](http://www.worldcommunitygrid.org/images/logo.gif)
YOU CANNOT WRITE TO THE SAME FILE CONCURRENTLY IN EITHER UNIX OR NT. THIS IS NOT DATASTAGE.
Unix buffers writes to files in chunks of bytes, this is THE WAY IT WORKS. So if you have simultaneous processes trying to write to the same file, you will NOT interleave rows, you with interleave BYTES. Carriage returns and linefeeds are data, not ROW DELIMITERS to the operating system. You will end up with GARBAGE data, with some lines of clarity.
On NT you get into sharing violation issues. You also can get the same problem of as Unix of delayed writes.
Unix buffers writes to files in chunks of bytes, this is THE WAY IT WORKS. So if you have simultaneous processes trying to write to the same file, you will NOT interleave rows, you with interleave BYTES. Carriage returns and linefeeds are data, not ROW DELIMITERS to the operating system. You will end up with GARBAGE data, with some lines of clarity.
On NT you get into sharing violation issues. You also can get the same problem of as Unix of delayed writes.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle