Dataset warning-expecting 2 segments, read 3 segments.

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
deepa.y
Participant
Posts: 56
Joined: Mon Nov 28, 2011 10:47 pm
Location: Bangalore

Dataset warning-expecting 2 segments, read 3 segments.

Post by deepa.y »

Hi,
I have a sequence in which two parallel jobs are called in sequential manner.
First job will append to a dataset.
Second job reads from the above appended dataset and write to a sequential file.
When i run this job normally,i dont see any warnings.
If i run the job in parallel instances of 2 ,i see the below warning
Dataset: /home/TestFolder/fac.ds, expecting 2 segments, read 3 segments.
I guess this is because the first instance is trying to read the dataset while the second instance is appending the dataset.
Correct me if i am wrong.
Please let me know how can i overcome this warning ?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Don't run them in parallel. Never mind the fact that it doesn't make any sense to do that since one job is creating the source data the other job needs.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Your analysis of the problem cause is correct. While datasets are very efficient for reading/writing data, they remain at heart sequential files (perhaps "sequential files on steroids" might be more fitting). They are not databases and, along with sequential files, are 1-Writer or N-Readers. There is no way to overcome the warning. Perhaps changing your design so that the second process creates and writes to a new dataset instead of appending, then performing the append at a time when no other process is using the original dataset.
Post Reply