Complex Flat File - Multiple Record Types (Records ID)

newerakb · Post by **newerakb** » Fri Jun 18, 2010 3:02 pm

I'm having an issue defining a CFF that I don't see addressed in any other threads I've seen. My input file has a header and a trailer, defined by "HDDR" and "TRLR". The data records between the header/trailer do not have a record identifier.

So on the "Records ID" tab, I can define my header with "HDDR" and my trailer with "TRLR", but I don't have any way of defining my data records. The only option I have for an operator ("Op" column) is "=".

Is there a way to set my record type as "all records that do not start with HDDR or TRLR"? I don't really want to split the header/trailer into a separate file, since they contain meta-information I need to validate the rest of the data with. I also cannot use a sequential file because the header and trailer records do not have the same record length as the data records.

Thanks

nagarjuna · Post by **nagarjuna** » Fri Jun 18, 2010 3:23 pm

why you want to use CFF stage ? Is it multi record format ?
You can strip header and trailer from the file by a transformer

newerakb · Post by **newerakb** » Fri Jun 18, 2010 4:15 pm

I don't want to strip the header and trailer, that's my point. I want to keep them because they have data that I need to compare to the rest of the data records.

Also, the header and trailer records aren't the same length as the data records, so I can't use a fixed-length file stage. I can't get it into a transformer without creating a stage to load the file in.

nagarjuna · Post by **nagarjuna** » Fri Jun 18, 2010 4:21 pm

you can use a sequential file stage and read it as a varchar . After that in transformer stage you can separate the header , data , trailer information . I believe you are going to take the data in header & trailer for reconcilaition .

ray.wurlod · Post by **ray.wurlod** » Fri Jun 18, 2010 5:03 pm

Why not have multiple outputs from the CFF stage to receive the different record types?

newerakb · Post by **newerakb** » Mon Jun 21, 2010 6:20 am

ray.wurlod wrote:Why not have multiple outputs from the CFF stage to receive the different record types? ...

That would be great, but I can't find a way to describe my records in CFF. Only my header and trailer have a record ID, the data records don't have any identifying ID. But the CFF requires a record ID, and there is no "everything else" check box like there is for constraints.

IBM Analytics Champion 2009 - 2020 · Post by **asorrell** » Mon Jun 21, 2010 7:50 am

Unfortunately you cannot use the CFF stage, precisely because of the fact you do not have a "Detail" indicator for the data records. The CFF stage does a wonderful job of separating out multiple record formats, but only when there is an "indicator" field in ALL the records (and in the same spot) to indicate record type.

nagarjuna's solution is the one I use in cases like yours. Just use constraints to route the data to different links for processing.

newerakb · Post by **newerakb** » Mon Jun 21, 2010 9:12 am

Yup, that's what I ended up doing. Thanks everyone for your help.