Job Aborting randomly, runs successfully also

vmcburney · Post by **vmcburney** » Sat Aug 09, 2003 6:22 pm

The error message will hopefully also display a list of field values somewhere in the message log, this may help you debug the problem. For example this is a normal record:
Id=2324
FirstName="Fred"
LastName="Fruit"
This is a record where a non numeric has been passed into a numeric field:
Id=AB124
FirstName="Fred"
LastName="Fruit"

Have a look through and see if you can spot a character put into a numeric field, the numeric fields are easy to find as the values are not enclosed in commas. This may help you debug the cause.

Vincent McBurney
Data Integration Services
www.intramatix.com

spracht · Post by **spracht** » Sun Aug 10, 2003 2:43 am

malathi,

we have the same problems sometimes and in our case it seems to be related to the 'preload file to memory' option' in hashed file stages, especially for 'larger' lookups (500.000 records and more). After disabling this option the 'Abnormal Termination' doesn't occur again.

Stephan

degraciavg · Post by **degraciavg** » Wed Aug 13, 2003 10:20 pm

I encountered this problem before with the Aggregator stage on 4.2. I think this stage has got a memory leak problem. I once filed a case with Ascential tech support but didn't pursue it because I was not able to send the test data to them... perhaps, you would want escalate your case to them...

Your error does not look like a memory-leak problem though, it seems more likely that some records you're aggregating are coming in as NULLs. In DataStage, any arithmetic operation involving NULL values will result in NULL, i.e., 1 + NULL = NULL. This scenario might have caused your problem.

Anyway, here's a tip: the best way to do aggregation is to do it in the database (with the right index/es) not in DataStage.

Cheers,
vladimir

vmcburney · Post by **vmcburney** » Thu Aug 14, 2003 12:31 am

Smart work, the aggregator stage is very dodgy in those old versions and worth avoiding. Another way to replace an aggregation stage is to load the data into a local Universe table then do a group by out of that table.

Vincent McBurney
Data Integration Services
www.intramatix.com

spracht · Post by **spracht** » Thu Aug 14, 2003 2:24 am

Strange, we are on DS 5.2 and encountered frequent 'Abnormal Termination' aborts when aggregating data within uv-stages. We worked around it using aggregator stages!!!

Stephan