Transformer output counts not totalling input count

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
dzdiver
Participant
Posts: 36
Joined: Tue May 25, 2004 8:55 am
Location: global

Transformer output counts not totalling input count

Post by dzdiver »

I have a job with a transformer which takes in 907,124 rows.
it has two outputs, both with constraint on non null char(1) column.
The 1st has constraint input.colX='5'
The other has constraint input.colX <> '5'

The output counts are 814,682 and 60,142 respectively, which add up to 874,824.
This is a difference of 32,300 rows compared with input rows.

There are no rejects. What can have happened to the other rows?

As a workaround I was thinking I could use just one of the constraints and mark the other as a reject rows constraint, but I wouldnt know which one of the = or <> is processing the correct number of rows.

Any ideas?
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

Probably nulls.
Mamu Kim
trammohan
Participant
Posts: 47
Joined: Thu Nov 13, 2003 12:47 pm

Re: Transformer output counts not totalling input count

Post by trammohan »

dzdiver wrote:I have a job with a transformer which takes in 907,124 rows.
it has two outputs, both with constraint on non null char(1) column.
The 1st has constraint input.colX='5'
The other has constraint input.colX <> '5'

The output counts are 814,682 and 60,142 respectively, which add up to 874,824.
This is a difference of 32,300 rows compared with input rows.

There are no rejects. What can have happened to the other rows?

As a workaround I was thinking I could use just one of the constraints and mark the other as a reject rows constraint, but I wouldnt know which one of the = or <> is processing the correct number of rows.

Any ideas?

Check the LOg in Director .. may be due to Nulls
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Wait, I want to play too... the difference is the number of rows where input.colX is null. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
dzdiver
Participant
Posts: 36
Joined: Tue May 25, 2004 8:55 am
Location: global

Post by dzdiver »

Yes you are all correct it was because the col was null :roll:

So does setting an input column's 'Nullable' property to 'No' have no effect ?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The nullable property is purely documentary within DataStage. It enforces nothing.
It serves as a valuable check for developers not to allow nulls to get through to columns in target tables that are defined to be not null.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

I guess my answer was too direct or too short. I wanted the shortest answer. How'd I do?
Mamu Kim
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Serks
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
dzdiver
Participant
Posts: 36
Joined: Tue May 25, 2004 8:55 am
Location: global

Post by dzdiver »

kduke wrote:I guess my answer was too direct or too short. I wanted the shortest answer. How'd I do?
"Likely" instead of "Probably" might have been shorter :)

Gold star to trammoham for the longest :lol:
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

ray.wurlod wrote:Serks
Tell them Ray man.
Mamu Kim
Post Reply