I have a job where I want to split 15 rows into 3 sets of 5 rows each.
My config file has 2 nodes. I would like 5 distinct rows to go into each of the
3 datasets. I wrote the following constraints for the links in a transformer:
link1: @PARTITIONNUM = 0 AND @OUTROWNUM < 6
link2: (@PARTITIONNUM = 0 AND @OUTROWNUM > 5) OR (@PARTITIONNUM = 1 AND @OUTROWNUM > 5)
link3: @PARTITIONNUM = 1 AND @OUTROWNUM < 6
link1 and link3 gets the proper values, i.e., proper values of @PARTITIONNUM and
@OUTROWNUM. I am also outputting these 2 system variables in each of the 3 links.
However, link2 is empty. When I set the link2 condition to "@OUTROWNUM > 0"
then all 15 rows go into that link along with proper 5 rows into link1 and link3.
Can anyone explain why link2 is not getting the proper rows?
@PARTITIONNUM AND @OUTROWNUM question
Moderators: chulett, rschirm, roy
Re: @PARTITIONNUM AND @OUTROWNUM question
Can't figure out why, but perhaps you can use this :
link1: @PARTITIONNUM = 0 AND @OUTROWNUM < 6
link2: @PARTITIONNUM = 1 AND @OUTROWNUM < 6
link3: make this the otherwise link
link1: @PARTITIONNUM = 0 AND @OUTROWNUM < 6
link2: @PARTITIONNUM = 1 AND @OUTROWNUM < 6
link3: make this the otherwise link
It took me fifteen years to discover I had no talent for ETL, but I couldn't give it up because by that time I was too famous.
@INROWNUM is the number of records that have been read from input. It will count up, the @OUTROWNUM for the link with @OUTROWNUM > 6 will not because no records are going down it.
@INROWNUM, @OUTROWNUM
1 0
2 0
3 0
4 0
5 0
6 0
At this point if you had the constraint as @INROWNUM > 6 records would go down that link and the value for @OUTROWNUM would change.
7 1
8 2
etc......you need to realise that @OUTROWNUM is specific to a link.....
@INROWNUM, @OUTROWNUM
1 0
2 0
3 0
4 0
5 0
6 0
At this point if you had the constraint as @INROWNUM > 6 records would go down that link and the value for @OUTROWNUM would change.
7 1
8 2
etc......you need to realise that @OUTROWNUM is specific to a link.....
Regards,
Nick.
Nick.
Well, that is right and that's exactly what makes your constraint definitions absolutely correct. Let's dive deeper :
15 records two nodes.
node0 = 7 recs
node1 = 8 records say.
link1: @PARTITIONNUM = 0 AND @OUTROWNUM < 6 hence first 5 records that go out of node 0 are allowed
similarly for link2. For link 3 put it as otherwise so that remaining 5 records go into it. Infact I tried it out and it worked. :D
15 records two nodes.
node0 = 7 recs
node1 = 8 records say.
link1: @PARTITIONNUM = 0 AND @OUTROWNUM < 6 hence first 5 records that go out of node 0 are allowed
similarly for link2. For link 3 put it as otherwise so that remaining 5 records go into it. Infact I tried it out and it worked. :D
It took me fifteen years to discover I had no talent for ETL, but I couldn't give it up because by that time I was too famous.