Need PX design for this scenario

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
HemaV
Participant
Posts: 63
Joined: Wed Jan 09, 2008 1:38 am
Location: Bangalore

Need PX design for this scenario

Post by HemaV »

Hi All,
I'm new to DS PX. Could you please suggest me the design for this scenario. I'm having huge records so wanted to build the job thinking about the performance also.

SQL Query:
SELECT * FROM ABC WHERE (aaa,bbb,ccc) IN (
SELECT aaa,bbb,ccc FROM ABC WHERE
STATUS = 'CLOSE' AND FLAG='Y' AND TRUNC((sysDate-STATDATE) / 365) >= 1
MINUS
SELECT aaa,bbb,ccc FROM DEF WHERE LVL <> 0
)

Needed DS PX design for this sql query.

Thanks in Advance,
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Put user-defined SQL into an appropriate stage type.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Nsg
Premium Member
Premium Member
Posts: 37
Joined: Thu Jan 26, 2006 1:21 pm

Re: Need PX design for this scenario

Post by Nsg »

I would split teh query -

perform 2 extracts -
for table ABC -
SELECT aaa,bbb,ccc FROM ABC WHERE
STATUS = 'CLOSE' AND FLAG='Y' AND TRUNC((sysDate-STATDATE) / 365) >= 1
for table DEF
SELECT aaa,bbb,ccc FROM DEF WHERE LVL <> 0

and the do Lookup / merge (based on volume) and reject the records the donot match .... the rejected data is the data you are looking for ... ignore what matched.

Hope this helps.

Ng
Jayanto
Participant
Posts: 41
Joined: Fri Feb 18, 2011 12:37 am
Location: Kolkata

Post by Jayanto »

@HemaV :: The simplest solution will be 'As Ray suggested'.... To build a user defined query, in an appropriate DS Stage.

Breaking in further, will prolong the reach to solution.
Regards,
Jayanto
prajesh
Participant
Posts: 7
Joined: Sun Apr 24, 2011 4:52 am

Post by prajesh »

hi,

both ABC,DEF tables are same database or different.if same DB you can prefer user defined query.
If both tables are from different db you can go through px design and you mentioned huge amount of data.
so you can better design with join stage using leftouter.


abc--->sort---->join(leftouter)---->target
|
|
sort
|
|
def
prajesh
Post Reply