DSXchange

Hi Ray,

I opened the performance analysis tool. I could see various charts but could not interpret them to arrive at a conclusion in terms of bottleneck.

Is there any document available on this tool apart from a short section available in Advanced parallel guide.

Thanks,

Shashank

Hi Ray, Thanks a lot. With careful examination of job score I'm able to relate what said with the actual score. By sequence I meant, any particular record in a given partition will be passed from operator to operator in the given sequence only. But that is obvious because we put stages in the same o...

Thanks Ray,

as the issue no longer exists, I would prefer to skip this for now.

Thanks,

Shashank

Hi, I was trying to understand the score dump of my PX job. My query is related to buffering. I read in Advanced guide that score dump provides information on where data is buffered. But I could not understand the meaning of an datasets and operators for buffering in the score dump i.e. Data Set exa...

Hi, Initially my job was running. Then I wanted eliminate operator combination and then I received below error, Lookup_EffDate,3: Fatal Error: Virtual data set.; output of "APT_LUTProcessOp in Lookup_EffDate": the record is too big to fit in a block; the length requested is: 139228, the ma...

Hi, I was trying to do Range lookup to a SCd type 2 table where I was trying to get values from reference link based on Transaction_Code/Transaction_Date combination. Since the unique key combination in reference link is Transaction_Code and Effective_date, it was must to define both as key columns ...

Hi Ray,

I check the score, APT_LUTProcessOp is running on all 4 nodes for both the lookups.

Thanks a lot for all the help.

Shashank

Hi,

I was looking for some help on lookup stage and found an interesting discussion on IO limited jobs.

How can I find if a job has performance limitations due to IO.

Thanks,

Shashank

Thanks Ray for all the help. The next question is, what can I do to run the lookup on all the nodes.

Thanks,

Shashank

Hi Ray, Thanks for your reply. We are on SMP systeam and I think this might be reason for it as I checked for other two and it is not the case. Execution mode for stages is Parallel and no resource pool constraints are added. Does this mean the lookup will always work on single node or only the look...

Hi, I have a PX job which is doing several lookups in two lookup stages. I got the Score Dump for the job. I'm pasting a portion of it below which refers to lookup part. I'm not sure if my interpretation is correct but I feel the lookup operation is happening on one node (out of 4 in config file). C...

Hi All, I tried few suggestions and here are the outcome: #Max attainable speed using select and dumping data in seq file is 5.5k row/sec Max attainable speed using sqlldr 'direct load' from is >10k row/sec. I have a question on this, why is there almost 4k row/sec drop when using datastage ? Can th...

@gateleys Thanks for your suggestion, speed increased from 90 to 140 using inter process. 140 is with all seq file as source and target. I'll post the numbers after replacing the source and targets with OCI stages. @Swapnil I'll try your suggestion @Ray yes the inter-process row buffering is enabled...

Swapnil

The OCI lookup will make the process more slower.


-
Shailesh

Yes all the OCI Stages. I even removed the source OCI with seq file.
And yes I meant preload to memory is enabled in Hashed files.


-
Shailesh