CFF vs Seq File stage

mydsworld · Post by **mydsworld** » Tue Oct 28, 2008 1:00 pm

Please let me know when the use of CFF is absolutely required and can not be done using Seq File stage.

ray.wurlod · Post by **ray.wurlod** » Tue Oct 28, 2008 1:43 pm

When you have any structured data (repeating groups, redefines), when you have packed decimal fields, just to name a couple.

mydsworld · Post by **mydsworld** » Tue Oct 28, 2008 1:53 pm

For packed decimals,
can not we do that in seq file stage using format option for Decimal, Packed = yes and setting EBCDIC

John Smith · Post by **John Smith** » Tue Oct 28, 2008 3:51 pm

yes for packed decimals you can use the seq file stage. CFF is mainly for files you get from mainframes i,e structured data.

ray.wurlod · Post by **ray.wurlod** » Tue Oct 28, 2008 8:39 pm

I meant packed decimals within structured data. Sorry for any confusion.

mydsworld · Post by **mydsworld** » Wed Nov 05, 2008 3:52 pm

Can we Convert a file from EBCDIC format to ASCII format with a design like this

Seq File with EBCDIC -> Seq File with ASCII

Also, do we need to specify 'Data Format = Binary' for the 'Seq File with EBCDIC'.

Thanks.

bcarlson · Post by **bcarlson** » Thu Nov 06, 2008 12:02 pm

About 90% of our input data from 100+ sources are from the mainframe and we do not use CFF anywhere, only SEQ file stage. There are some circumstances where CFF would have been handy, but its functionality in PX is not complete compared to Server - mainly in the handling of redefines or multiple record types in single file.

If the data has occurences (repeating groups) we use vectors in the import schema.
We remove (or ignore) column level redefines or have the source remove them.
If the redefines are used to represent multiple record types, then we import the file by only defining the first X bytes - enough to provide natural keys and record types. The rest of the record is left a one big raw field. Then we split the output with a filter based on record type and the output of each of those streams uses the column import to apply a record-type specific layout to the raw field.

Hope this helps.

Brad

vkhandel · Post by **vkhandel** » Fri Feb 06, 2009 1:44 am

Hi Brad

We are doing exactly as the last point of your's (providing enough natural keys and record types), but we are facing an issue due to the presence of "packed decimal" values in the raw field of some record type.

Could you please suggest, if there is any property in Column Import stage which will unpack the "packed decimal" values.

bcarlson · Post by **bcarlson** » Fri Feb 06, 2009 9:45 am

You unpack a decimal in the column import the same as you would with an import. Your record should already be specified as binary EBCDIC. Set your import field to Decimal with the approriate length and scale and then set Packed to Yes (Packed is an option for the Decimal datatype).

Brad.