OCI Stage acting wierd

ak77 · Post by **ak77** » Sun Dec 18, 2005 1:42 am

Hi everybody,

In one of my job, I am getting data from sequential file and using aggregator stage to get the min of a group of values and transformer to update a table using OCI stage. 4 sequential files are merged and sorted to feed the sequential using before-job sub-routine ExecSH command

The jobs worked fine for small dataset, i ran the job for the whole dataset, the job aborted with mutex error
So i removed the duplicates while sorting the data for sequential file

I ran the job again, now it finished the job but the update statement was not found in the log.This job also ended much faster but found so many rows not being updated

Can someone explain me this?

Thanks
Kishan

Can

ak77 · Post by **ak77** » Sun Dec 18, 2005 2:10 am

Hi
I changed the job to see if all the data is going correct
Now instead of updating to the Table, i just loaded to a sequential file

I found something strange
The number rows coming into the Aggregator is 12814
The number of rows going out of the Aggregator is 12054
This is a direct map from the aggregator to the sequetial file through a transformer with 2 extra fields added
The output of the transformer has 13180 rows

The job aborted with the message

At row 13990,Row out of sequence message

i am using sort command in the before job subrouiting

Code: Select all

sort -t'|' -u -T #tempDirectory# -k1,1 -k2,2 -k3,3 file1 file2 file3 file4 > outfile

Also i am sorting the input in the aggregator in the same order 1 , 2 and 3

Why am i getting this error

Help me with this

Thanks
Kishan[/quote]

ray.wurlod · Post by **ray.wurlod** » Sun Dec 18, 2005 2:26 am

You might need the -n option for sorting numeric fields with the sort command, otherwise they're sorted as strings.

ArndW · Post by **ArndW** » Sun Dec 18, 2005 2:39 am

You have more rows coming out of your Transform stage than you have going in. What are you doing in this stage? Are any of the two fields you add part of the sort key/order?

ak77 · Post by **ak77** » Sun Dec 18, 2005 2:39 am

Thanks Ray,

2 of the field are char, 1 varchar, and 2 number and still i need to give the -n option
Does it make sense to sort again in the aggregator stage?

Thanks again

Kishan

ak77 · Post by **ak77** » Sun Dec 18, 2005 2:42 am

Well Arndw,

The output from aggregator has 5 fields
I am dropping one field
I am renaming one of the field
Adding one column to populate with 'Y'

These are direct maps

I am not doing anything else in the transformer

Kishan

ak77 · Post by **ak77** » Sun Dec 18, 2005 2:53 am

Ray -

I added the -n option
It makes sense
I thought it to be something different

The job finished successfully

Thanks everybody

Kishan