CFF Stage issues

[clarcombe]

I have to import a Cobol text file and structure into Datastage.

The structure has multiple occurs in, some of which are nested

Code: Select all

Fields
     Occurs 30
     Occurs 6
     Occurs 140
         Occurs 5
Fields

I am experiencing several issues

1) I had to import the decimals e.g. +00568 as strings as they were blank otherwise. Is this usual ?

2) I seem to be getting a cartesian product as I have about 35 rows, which when read, creates about 160,000 lines. I can only guess this must be 35x30x140. Again is this usual ?

3) I am having to read the sub records into one field in a separate file with the key and then reprocess them to extract the fields, then link them to the header file. Is there a better way ?

4) Are there any better alternatives than using the CFF file ?

I am unable to use the enterprise version as we only have the 7.5 server license.

Thanks

[ray.wurlod]

You could read the entire line as a single string using Sequential File stage, and insert CRLF strings where you want them (and maybe commas) before writing back to a file. Then read from that file.

For example, if you have PIC XX OCCURS 30 you can read a 60 character string. Use Fmt() or Fold() function to split it into two-character substrings and then convert the delimiter (@TM for Fmt() or @FM for Fold()) into CRLF using Ereplace() function. Tip: initialize stage variable svCRLF to Char(13):Char(10) and use that as the "to" argument in Ereplace()

[bhargav_dd]

There is no alternative option than using CFF stage as it reads these records and does the required output

[ray.wurlod]

I disagree, as I explained above.

[clarcombe]

Thanks Ray,

An interesting alternative which I began to implement then stopped.I really want to avoid the counting and substring techniques as the structure may well change and would make it difficult to maintain.

So it appears then, that the multiplication of lines in CFF is normal. Luckily the CFF runs rapidly so the job completes quickly.

Regards

[clarcombe]

This is a note to all those who have speed issues with the CFF. If there are multiple nested structures in the file, don't try and take them all out in one go. Use the row key and take them out separately and rejoin them

Thus
Key
Structure A
Structure B
Structure C

Use three CFF stages instead of one to create three files
Key
Structure A
Key
Structure B
Key
Structure C

Using one CFF stage will make a cartesian product of the result
i.e. Rows x Structure A occurences x Structure B occurences x Structure C occurences.