JOb Running for Hours

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
vamsi.4a6
Participant
Posts: 334
Joined: Sun Jan 22, 2012 7:06 am
Contact:

JOb Running for Hours

Post by vamsi.4a6 »

Hi all,

Job is running for a long time


Orcale connector(source)-->aggregator--->transformer-->Orcale connector(Target)


source oracle connector consists of query using 4 tables joined based upon Join Keys.

Enabled partition=No, so the oracle connector is running in the sequentail Mode.
In aggregator 6 group by keys are present.Is there any way to improve the performance of Job by selecting Enabled partition=Yes or some other option?
chandra.shekhar@tcs.com
Premium Member
Premium Member
Posts: 353
Joined: Mon Jan 17, 2011 5:03 am
Location: Mumbai, India

Post by chandra.shekhar@tcs.com »

I think enabling Hash partitioning in the source would help.
And what about the partitioning at the target level ?
Also how much data we are talking about here ?
Thanx and Regards,
ETL User
vamsi.4a6
Participant
Posts: 334
Joined: Sun Jan 22, 2012 7:06 am
Contact:

Post by vamsi.4a6 »

@Thanks for the input

1)Target is key partition

2)I am running the JOb for 63 iterations using Loop activity in
Sequence.For each iteration the volume of input data is 50 Lakhs approximately

Clarification:1)Is select Enabled partition=YES and Hash partitioning will effect aggregator functionality?
2)Hash partitioning,oracle partitons,rowid are datastage partitions or oracle partitions?
chandra.shekhar@tcs.com
Premium Member
Premium Member
Posts: 353
Joined: Mon Jan 17, 2011 5:03 am
Location: Mumbai, India

Post by chandra.shekhar@tcs.com »

I think It will not affect the aggregator performance, as I have used the same in DB2. And for your 2nd point, hope this might help.

<corrected>
Thanx and Regards,
ETL User
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

chandra.shekhar@tcs.com wrote:(copy-pasted from IBM Information center)
:!: Please do not that. Link to articles in the Information Center, do not post their contents here wholesale.
-craig

"You can never have too many knives" -- Logan Nine Fingers
chandra.shekhar@tcs.com
Premium Member
Premium Member
Posts: 353
Joined: Mon Jan 17, 2011 5:03 am
Location: Mumbai, India

Post by chandra.shekhar@tcs.com »

Aye Aye Captain.. :oops:
Thanx and Regards,
ETL User
vamsi.4a6
Participant
Posts: 334
Joined: Sun Jan 22, 2012 7:06 am
Contact:

Post by vamsi.4a6 »

@chandra

Thanks for the input.one small clarification,If i am using multiple tables Joined based upon Join Keys or sub queries in source oracle connector I am not sure which table to use under option-Table name for partitioned reads?
chandra.shekhar@tcs.com
Premium Member
Premium Member
Posts: 353
Joined: Mon Jan 17, 2011 5:03 am
Location: Mumbai, India

Post by chandra.shekhar@tcs.com »

vamsi, I dont think that you'll be able to to do that.
If your query has multiple tables in it, then its difficult to achieve partitioned read.
But you can test it at your end by running the job and taking the driving table of your query for partitioned reads and match the count. Give it a try.
Thanx and Regards,
ETL User
vamsi.4a6
Participant
Posts: 334
Joined: Sun Jan 22, 2012 7:06 am
Contact:

Post by vamsi.4a6 »

vamsi.4a6 wrote:If i am using multiple tables Joined based upon Join Keys or sub queries in source oracle connector I am not sure which table to use under option-Table name for partitioned reads?
Could anybody please clarify this doubt?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You. As noted. Assuming your tables are actually partitioned.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply