Page 1 of 1

Aggregation

Posted: Fri Jul 17, 2009 10:10 am
by karthi_gana
All,

I have created a job which will fetch data from a table and load it in two files.

format:

fund_id
fnd_dt
fund_cd
qty
price

i will fetch the above information and store it in two file based on the below logic.

1) If fnd_cd = 'C' then i will aggregate it and store it in a file
2) if fnd_cd <> 'C' then i will store all the entries in another file.

say for example,

i have like below

1 17/jul/2009 T 22 25000
1 17/jul/2009 C 2 500
1 17/jul/2009 C 2 100

1 17/jul/2009 D 13 4566
1 17/jul/2009 E 12 4440
1 17/jul/2009 Q 20 5800

I will aggregate the above bold one and store it in a seperate file say for example file2.

my problem is all the values are coming in file2.

i.e
first row is aggregated value
remaining rows also appended from the second row onwards.

But the job shows only one row transferred. How it is possible?
where i am doing mistake?

Posted: Fri Jul 17, 2009 10:14 am
by datskosaraju
You didn't give us your job design(flow) to find what's wrong!!!

Re: Aggregation

Posted: Fri Jul 17, 2009 10:15 am
by karthi_gana
I have also created an constraint saying if fnd_cd = 'C' then move it to file2
if fnd_cd <> 'C' then move it to file1.

Posted: Fri Jul 17, 2009 10:17 am
by karthi_gana
datskosaraju wrote:You didn't give us your job design(flow) to find what's wrong!!!
what you are expecting? can you tell me? i am not gettting you.

Posted: Fri Jul 17, 2009 10:21 am
by chulett
Explain your job design - basically list the stages used and what role each is playing in the job.

Posted: Fri Jul 17, 2009 10:25 am
by datskosaraju
chulett wrote:Explain your job design - basically list the stages used and what role each is playing in the job. ...
Thanks Craig!! That was exactly the same thing I was talking about before

Posted: Fri Jul 17, 2009 10:28 am
by karthi_gana
chulett wrote:Explain your job design - basically list the stages used and what role each is playing in the job. ...


file1
|
|
sequential file --> transformer --> aggregation ---> file2


in the transformer , i wrote a constrain like
iif fnd_cd <> 'C' then file1
f fnd_cd = 'C' then file2

Posted: Fri Jul 17, 2009 10:38 am
by priyadarshikunal
Unless I am missing something in your posts it seems like constraint you have posted is not complete or you have mentioned same filename as file1 and file2 in append mode or some thing is missing from your explanation.

can you post your exact constraint syntax? also are you using otherwise link as your input to aggregator stage?

Posted: Mon Jul 20, 2009 4:10 pm
by ray.wurlod
You need TWO outputs from the Transfomer stage.

You need to write TWO files.

You can put them together afterwards.