DSXchange

Hi All, In one of my datastage jobs the final output is a file set.A dataset is a input here and after some transformation the fileset is created. Though there are 8 nodes in the datastage server in only three nodes records are exported but in the other 5 nodes 0 records are exported. The log messag...

Hi, In my jobs there many joins and lookups.I want sort the main and reference data . For main link the no. of records varies from 200000 to 60000000 for each join. For reference link no. of records varies from 20000 rows to 20000000. I want to know which one is preferable external sort stage or the...

Hi ,

Thanks all of you for your advices.
I got clear and satisfactory answers.How can i close this topic?

In this case the switch stage is exactly tailored to what you want to do and will be more efficient. If you have a large percentage of records that are dropped then the filter stage will be more appro ... Hi, Thank you very much for your advice. Can you plz explain me why switch stage is preferable...

Hi, I will have to segregate the records from a dataset to 14 file sets depending upon the 14 different values in a column. The number of records are 82460040.The records are not evenly distributed on all the output links.That means every output link does not carry same no. of records. Can u plz adv...

Basically we are handling huge amont of data every day(around 300GB!) and it is getting larger and lager every month. In most of the jobs the dataset is the first stage and final o/p stage i.e the output dataset of one job is acting as a input to the next job. In the jobs there are mainly join and t...

Hi,

How can i ensure that records of dataset are evenly distributed
on all the ds server nodes?Do i need to make some changes in

conf file?

Regards

Avik Dasgupta

Hi , I have checked the mount points.Data is well distributed accros all the 8 nodes.One thing i wish to inform that run time column propagation option is enabled.Is it delaying the read process? Its not about usage! try and find the mount.. Also when you say 8 node are all the 8 nodes used well and...

balajisr wrote:How many rows do you have in each partition?
What is your partition count?
Post your job design.You need to give more details.

Hi,

The partition count is 8 and i have checked the filesystem,the memory usage is ok.

ray.wurlod wrote:What are your parallel job tuning credentials? That is, why did they give you the task? How much experience do you have in this area?

I am very new to datastage.I developed some parallel jobs in last two
months.

Hi all,

I have been assigned to tune some parallel jobs.
I observed in some jobs reading from dataset is taking huge time.

Pls advice me to increase no. of rows/sec while reading from dataset.

thanks and regards

Avik Dasgupta