Problem with Aggretaor

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
Das
Participant
Posts: 87
Joined: Tue Oct 24, 2006 9:58 pm
Location: india

Problem with Aggretaor

Post by Das »

Hi,
When i am using the transformer i am getting the follwing error in director
Abnormal termination of stage ABCD..Aggregator_11 detected

I am taking two columns col1 and col2 from the source and in aggregator
group by col1 and max(col2)

passing the same to u/p file

Please help me out.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Do you have sorted input? Can you explain your job design a little bit more clearly. Do you have any more error messages? details man details.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

What's your volume? As noted, I'lll wager you need to sort your data and then assert that in the stage.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Das
Participant
Posts: 87
Joined: Tue Oct 24, 2006 9:58 pm
Location: india

Post by Das »

DSguru2B wrote:Do you have sorted input? Can you explain your job design a little bit more clearly. Do you have any more error messages? details man details.
No other error messeges,Data Volume is 3 crores of rows.
From input informix stage ,I am using simpe simple SQL Stament

Select col1,col2
From tab1

Then pass to transformer
Apply some constrints for filtering

Then pass to aggrator
Doing group by on col1 and max on col 2
Then agin pass to tranformer to do look up then to target.
the aggregator ouput link it is showing green colour and showing 0 rws proceesed,in put link it is showing some rows but it is in red.
Some times the error messege is as follows
ABC..Aggregator_12: yxy
Please help me
Das
Participant
Posts: 87
Joined: Tue Oct 24, 2006 9:58 pm
Location: india

Post by Das »

chulett wrote:What's your volume? As noted, I'lll wager you need to sort your data and then assert that in the stage. ...
3-6 crores rows of data.No soring i have used.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

We keep advising you to use sorted data. 30 million rows is not too many. Even a UNIX sort can manage this many. The Aggregator stage with unsorted input will almost always abort with this much data, but will not with sorted input.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Specify an order by in your sql or use an explicit sort stage to sort the data. Let the aggregator know how the data is sorted.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Das
Participant
Posts: 87
Joined: Tue Oct 24, 2006 9:58 pm
Location: india

Post by Das »

ray.wurlod wrote:We keep advising you to use sorted data. 30 million rows is not too many. Even a UNIX sort can manage this many. The Aggregator stage with unsorted input will almost always abort with this much data, ...
Thanks for the information.
Some of my tables contains 30 crores of rows.I have used the sort option in the input of aggrator stage on one column

I am getting the error
ABC..Aggregator_11: Row out of sequence
At row 89, link "DSLink14"
Row out of sequence
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The error is pretty self-explanatory. You've told the Aggregator to expect rows in a certain order and it found a row 'out of sequence' - one that violated your alleged order. Correct it.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Das
Participant
Posts: 87
Joined: Tue Oct 24, 2006 9:58 pm
Location: india

Post by Das »

chulett wrote:The error is pretty self-explanatory. You've told the Aggregator to expect rows in a certain order and it found a row 'out of sequence' - one that violated your alleged order. Correct it.
I have used the same order from tranformer which populate data to aggragator

Transformer
col1
col2
Aggragator input
col name - sort - sort order
col1 -- 1 -- Ascending
col2 -- 2 -- Ascending
Aggragator output
col name-- group -- Derivation
col1 -- *[Grup by this col] -- --
col2 -- -- -- max(col2)

The above are the details,Please suggest.
nick.bond
Charter Member
Charter Member
Posts: 230
Joined: Thu Jan 15, 2004 12:00 pm
Location: London

Post by nick.bond »

But you haven't sorted the data! Add sort stage between tfm and agg.
Regards,

Nick.
Post Reply