Page 1 of 1
Transformer output counts not totalling input count
Posted: Thu Sep 29, 2005 4:02 am
by dzdiver
I have a job with a transformer which takes in 907,124 rows.
it has two outputs, both with constraint on non null char(1) column.
The 1st has constraint input.colX='5'
The other has constraint input.colX <> '5'
The output counts are 814,682 and 60,142 respectively, which add up to 874,824.
This is a difference of 32,300 rows compared with input rows.
There are no rejects. What can have happened to the other rows?
As a workaround I was thinking I could use just one of the constraints and mark the other as a reject rows constraint, but I wouldnt know which one of the = or <> is processing the correct number of rows.
Any ideas?
Posted: Thu Sep 29, 2005 5:49 am
by kduke
Probably nulls.
Re: Transformer output counts not totalling input count
Posted: Thu Sep 29, 2005 6:43 am
by trammohan
dzdiver wrote:I have a job with a transformer which takes in 907,124 rows.
it has two outputs, both with constraint on non null char(1) column.
The 1st has constraint input.colX='5'
The other has constraint input.colX <> '5'
The output counts are 814,682 and 60,142 respectively, which add up to 874,824.
This is a difference of 32,300 rows compared with input rows.
There are no rejects. What can have happened to the other rows?
As a workaround I was thinking I could use just one of the constraints and mark the other as a reject rows constraint, but I wouldnt know which one of the = or <> is processing the correct number of rows.
Any ideas?
Check the LOg in Director .. may be due to Nulls
Posted: Thu Sep 29, 2005 7:08 am
by chulett
Wait, I want to play too... the difference is the number of rows where input.colX is null.
Posted: Thu Sep 29, 2005 9:18 am
by dzdiver
Yes you are all correct it was because the col was null
So does setting an input column's 'Nullable' property to 'No' have no effect ?
Posted: Thu Sep 29, 2005 4:56 pm
by ray.wurlod
The nullable property is purely documentary within DataStage. It enforces nothing.
It serves as a valuable check for developers not to allow nulls to get through to columns in target tables that are defined to be not null.
Posted: Thu Sep 29, 2005 8:53 pm
by kduke
I guess my answer was too direct or too short. I wanted the shortest answer. How'd I do?
Posted: Thu Sep 29, 2005 9:16 pm
by ray.wurlod
Serks
Posted: Fri Sep 30, 2005 5:03 am
by dzdiver
kduke wrote:I guess my answer was too direct or too short. I wanted the shortest answer. How'd I do?
"Likely" instead of "Probably" might have been shorter
Gold star to trammoham for the longest
Posted: Fri Sep 30, 2005 6:49 am
by kduke
ray.wurlod wrote:Serks
Tell them Ray man.