Loading flat files with multiple record types in different t

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kumarjit
Participant
Posts: 99
Joined: Fri Oct 12, 2012 7:47 am
Location: Kolkata

Loading flat files with multiple record types in different t

Post by kumarjit »

I need to read a flat file with multiple record types and load data corresponding to each record type in separate oracle tables.
I'm yet to use CFF stage but I've learnt from many sources that CFF's do read files with multiple record schema.
The most challenging aspect of this job(to me) is that the number of distinct record types in the file may change from time to time, hence the number of tables where data is to be inserted should be dynamic (depending upon the number of distinct record types).

The source extract basically consists of data of multiple prepaid services types, whose attributes change with the service type, like prepaid GSM, WIMAX, Broadband or Dial-ups.

Please help.

Thanks in advance,
Kumarjit.
Pain is the best teacher, but very few attend his class..
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You're not going to be able to design a job that dynamically adjusts the number of target tables. You can, however, design one that targets all known target tables where not all will actually receive data during any given run. And yes, the CFF can handle multiple record types but there are other mechanisms if that's all you need to handle, meaning things you don't need that the CFF also does like EBCDIC to ASCII conversion, packed field handling, etc etc. Basically it's primarily meant for mainframe COBOL data.

Records could be read as a single long string and then depending on the record type split out to the appropriate target and then parsed via (let's say) a Column Import stage for that particular record layout.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply