Hello,
I am using DS 8.1 Version on Unix (AIX) platform. We have a requirement for a project where in the source and Targets are Sequential files, with transformers, lookups, sort (HASH partition on 4 columns) and Remove Duplicates as intermediate stages.
The source is a Fixed width file with 6-8GB size limit and approximately 70Million Records (58 Character Length, for each record).
I can put in an approximate size for the output files being generated (8GB approx),
However am interested to know is what would be the approximate size (maximum value) of Scratch Disk and Resource Disk that the DS Server would need for the same, and the total file system size requirement? Please do let me know how we arrive at the conclusion/stats for the same?
Resource Estimates for Scratch and Resource Disk
Moderators: chulett, rschirm, roy
Resource Estimates for Scratch and Resource Disk
Last edited by vvsaucse on Fri Apr 15, 2011 9:58 am, edited 2 times in total.
Subbu
TCS BAngalore
TCS BAngalore
I think your scratch disk estimation really depends on your current and future job design.
If you sort or merge the data for instance, you'll use the scratch space. So your job design directly affects how much of that space is used.
You also have other jobs running in your environment that might also use that scratch space.
My advice... go big.
It is true that a $$$ amount for hardware should not affect your code design, but for scratch space... more is better.
If you sort or merge the data for instance, you'll use the scratch space. So your job design directly affects how much of that space is used.
You also have other jobs running in your environment that might also use that scratch space.
My advice... go big.
It is true that a $$$ amount for hardware should not affect your code design, but for scratch space... more is better.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Was trying to use the Resource Estimate option , but getting the below error messages for the same job, which is running to completion with out this option., Would help if anyone can shed some light on the same
main_program: Current heap size: 1,554,287,264 bytes in 6,709,915 blocks
main_program: Fatal Error: Throwing exception: APT_BadAlloc: Heap allocation failed.
main_program: Current heap size: 1,554,287,264 bytes in 6,709,915 blocks
main_program: Fatal Error: Throwing exception: APT_BadAlloc: Heap allocation failed.
Subbu
TCS BAngalore
TCS BAngalore