Data Analysis ???

Developer9 · Post by **Developer9** » Wed Oct 12, 2011 8:50 pm

Hi,

How to perform data validation on fields that are changing weekly ?

I have created delta files (Sequential files)USING " cdc stage" but these are big enough as full size volume (5GB) to cause crash on server

As a Data Stage Developer .what is the scope ???

ray.wurlod · Post by **ray.wurlod** » Wed Oct 12, 2011 8:56 pm

5GiB isn't a lot, unless your server lacks "large file support". Contact your system administrator.

Developer9 · Post by **Developer9** » Mon Dec 12, 2011 11:58 am

@ray

There is a limitation on server lacks "large file support"

Environment:Data Stage 7.5.2 Parallel Edition

Code: Select all

compare two datasets(Today's records and previous days records) >>>CDC>>>target delta (sequential file)

We are looking to fix this issue by modifying the job job such a way that it splits the target delta(weekly)file into two:

Category 1:File that contain fields that dont change often ..(size is less supposed to be) "

Category 2:File that contain fields that change often (weekly)
(size is larger compare to category 1 files)"

How to modify the job design /code in order to generate (two)separate files for this fields ???

Is there any approach to minimize the file size ???

Thanks for the input