@PARTITIONNUM AND @OUTROWNUM question

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
splayer
Charter Member
Charter Member
Posts: 502
Joined: Mon Apr 12, 2004 5:01 pm

@PARTITIONNUM AND @OUTROWNUM question

Post by splayer »

I have a job where I want to split 15 rows into 3 sets of 5 rows each.
My config file has 2 nodes. I would like 5 distinct rows to go into each of the
3 datasets. I wrote the following constraints for the links in a transformer:

link1: @PARTITIONNUM = 0 AND @OUTROWNUM < 6
link2: (@PARTITIONNUM = 0 AND @OUTROWNUM > 5) OR (@PARTITIONNUM = 1 AND @OUTROWNUM > 5)
link3: @PARTITIONNUM = 1 AND @OUTROWNUM < 6

link1 and link3 gets the proper values, i.e., proper values of @PARTITIONNUM and
@OUTROWNUM. I am also outputting these 2 system variables in each of the 3 links.
However, link2 is empty. When I set the link2 condition to "@OUTROWNUM > 0"
then all 15 rows go into that link along with proper 5 rows into link1 and link3.
Can anyone explain why link2 is not getting the proper rows?
sud
Premium Member
Premium Member
Posts: 366
Joined: Fri Dec 02, 2005 5:00 am
Location: Here I Am

Re: @PARTITIONNUM AND @OUTROWNUM question

Post by sud »

Can't figure out why, but perhaps you can use this :

link1: @PARTITIONNUM = 0 AND @OUTROWNUM < 6
link2: @PARTITIONNUM = 1 AND @OUTROWNUM < 6
link3: make this the otherwise link
It took me fifteen years to discover I had no talent for ETL, but I couldn't give it up because by that time I was too famous.
nick.bond
Charter Member
Charter Member
Posts: 230
Joined: Thu Jan 15, 2004 12:00 pm
Location: London

Post by nick.bond »

@OUTROWNUM is specific to a link. What you are saying is "Only send data down here if more than 6 rows have already gone down here" which obviously can't happen.

You need to use @INROWNUM
Regards,

Nick.
splayer
Charter Member
Charter Member
Posts: 502
Joined: Mon Apr 12, 2004 5:01 pm

Post by splayer »

nick.bond, interesting! I didn't think of that. That's why I was getting 0 rows but how is using @INROWNUM going to help? Wouldn't that be the same scenario?
nick.bond
Charter Member
Charter Member
Posts: 230
Joined: Thu Jan 15, 2004 12:00 pm
Location: London

Post by nick.bond »

@INROWNUM is the number of records that have been read from input. It will count up, the @OUTROWNUM for the link with @OUTROWNUM > 6 will not because no records are going down it.

@INROWNUM, @OUTROWNUM
1 0
2 0
3 0
4 0
5 0
6 0

At this point if you had the constraint as @INROWNUM > 6 records would go down that link and the value for @OUTROWNUM would change.

7 1
8 2

etc......you need to realise that @OUTROWNUM is specific to a link.....
Regards,

Nick.
sud
Premium Member
Premium Member
Posts: 366
Joined: Fri Dec 02, 2005 5:00 am
Location: Here I Am

Post by sud »

Well, that is right and that's exactly what makes your constraint definitions absolutely correct. Let's dive deeper :

15 records two nodes.

node0 = 7 recs
node1 = 8 records say.

link1: @PARTITIONNUM = 0 AND @OUTROWNUM < 6 hence first 5 records that go out of node 0 are allowed
similarly for link2. For link 3 put it as otherwise so that remaining 5 records go into it. Infact I tried it out and it worked. :D
It took me fifteen years to discover I had no talent for ETL, but I couldn't give it up because by that time I was too famous.
Post Reply