CFF drawbacks in Parallel Jobs

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
gsherry1
Charter Member
Charter Member
Posts: 173
Joined: Fri Jun 17, 2005 8:31 am
Location: Canada

CFF drawbacks in Parallel Jobs

Post by gsherry1 »

Hello Forum,

It seems to me that the CFF is somewhat less useful in Parallel Jobs than in Server Jobs. The majority of files I receive from MVS have multiple record types and redefined fields. In server edition I would handle each record type on a separate output link of the CFF. It would appear to me that reading multiple record layouts in CFF is not really feasible, particularily when the datatype differs between record types (PIC 9 vs COMP-3).

The Seqeuential Flat File stage seems to have better EBCIDIC and packed features, but these also don't seem very useful when there is more than one record type. It seems that in order to parse such files in Parallel jobs I am forced to first write a job that parses source file witch schema like this:

record (
rec_id:decimal[3];
remainder:raw[500];
)

Then after splitting by rec_id, land the file and reread with more specific schema to each record. Is this what others are doing?

Given that the Sequential flat file has both EBCDIC and packed features, it seems the useful feature of the CFF in parallel jobs is to have some advanced flattening techniques for arrays and occur clauses. If those features are not necessary is it recommended to use the Sequential Flat File? Is Sequential flat file faster than CFF?

Your input is appreciated.


Greg
ukyrvd
Premium Member
Premium Member
Posts: 73
Joined: Thu Feb 10, 2005 10:59 am

Re: CFF drawbacks in Parallel Jobs

Post by ukyrvd »

gsherry1 wrote: Then after splitting by rec_id, land the file and reread with more specific schema to each record. Is this what others are doing?
Yes .. we have similar requirement and we followed similar technique
thank you
- prasad
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

It's probably worth either asking the vendor whether it's changed in the Hawk release or putting in an enhancement request. There's a forum here for the latter, or you can do it through your support provider.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply