DSXchange

Hi All,

I am loading 1.2 billion to my fact table, I have used LOOKUP stage to join all the dimentions. When I kicked off the job it has started at very good seed but when it had reached to .5 billion, It got aborted ...Log file showing UNIX Signal SIGKILL 9...that means that job has aborted forcefully....And when i looked into status of unix box....one folder with /Dev/fstst....it is showing used memory 100%...My source and target is DB2.

Can any one help me....what could be the reason

Thanks in Advance

stiwari

That might be your temp directory. The Scratch will get used during run time. Are you using lookup fileset or sparse lookup?
You may need to increase the disk space for the directory.

Kumar,

I am using lookup type as normal......Is it really matter that i have to use lookup type that won't be using Scratch file this much....


stiwari

Technically, of course, every UNIX signal can be generated by a kill command. In this case a process has - as you noted - been forcefully terminated with a SIGKILL signal (kill -9).

Yes, the Lookup stage does use scratch space if it needs to. The Lookup stage in normal mode is (generates) a composite operator - one to read in ALL the lookup data (for example into a virtual Data Set), the second to perform the actual lookup processing. Building a virtual Data Set consumes memory.

You may need to find a better strategy for combining your Data Sets.

Lookup will be always efficient when the lookup data can be made to fit into memory. Try to increase the scratch space else try out some other strategy like converting lookup into join.

Can anyone tell, for what purpose the folder /dev/fstst/ is being used? Is it used for some temporary loading by lookup stage?

Check in your DS Adminstrator setting for environmental varialbe TEMP_DIR, if it is pointing to this directry, it is used as scratch space.
Else check your Config file, you can find if it is been used as one of the resource pool.

Your UNIX administrator will be able to tell you whether that is a device that you are not supposed to be using, or part of the file system. To me the name "fstst" suggests "file system test" which in turn suggests that you may not be supposed to be there.