Hi,
I have a parallel job with a transformer stage which redirectes the input data to three different links based on some constraints.
The input data is as follows:
colA colB
1 100
1 null
5 541
7 null
The constraint on output links are as follows:
link1: colA = 1
link2: colA=1 Or colA=3
link3: otherwise link
The order of links in the link ordering tab is as follows:
link1
link2
link3
Now, in the derivation of link1, I'm performing a calculation as colB+1 and in link2, I mapping the columns from the input without any manipulations to the output. Since I have not performed any null handling in colB, I expect records to be dropped from link1 and records to be processed in link2.
But, here is what is happening:
link1: 0 rows (records getting dropped as null handling is not done) -- This is perfect
link2: 0 rows ??? I don't know why this is happening
link3: 2 rows -- This is expected behavior
Can anyone please tell me why does link2 in this case drop records?!!
-Deepti
Dropped Records in transformer Stage
Moderators: chulett, rschirm, roy
I would never force this type of error - doing an operation on a nullable field. Add IsNull() or similar handling to your job; link1 should have the constraint "ColA=1 AND NOT(IsNull(ColB))", link2 should be "ColA=1 OR ColA=3".
To your issue, what happens when the output of link2 is just ColA and ColB is not used?
To your issue, what happens when the output of link2 is just ColA and ColB is not used?
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Are you using stage variables? These are currently implicitly not nullable.
News from IOD 2009 is that they're going to be a bit less strict with some of the things in the Transformer stage in a future release, one of those things will be better tolerance of nulls in functions, expressions and stage variables.
News from IOD 2009 is that they're going to be a bit less strict with some of the things in the Transformer stage in a future release, one of those things will be better tolerance of nulls in functions, expressions and stage variables.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.