Complex Flat File-Occurs clause Issue

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
avi21st
Charter Member
Charter Member
Posts: 135
Joined: Thu May 26, 2005 10:21 am
Location: USA

Complex Flat File-Occurs clause Issue

Post by avi21st »

Hi

I am loading metadata definitiion from the cobol copy to my Complex Flat File. The copbol copy book has a structure of one of the field as OCCURS ........
DEPENDING ON ..... INDEXED BY .....
My question is How to define this field? Should it be a GROUP. How to define the range in the Occurs clause. I have seen that we can give only one value in the Occurs clause of the Complex Flat file. Also how to give the Indexd by clause

Please help me in this regard

Thanks
Avishek Mukherjee
Data Integration Architect
Chicago, IL, USA.
avi21st
Charter Member
Charter Member
Posts: 135
Joined: Thu May 26, 2005 10:21 am
Location: USA

Re: Complex Flat File-Occurs clause Issue

Post by avi21st »

avi21st wrote:Hi

I am loading metadata definitiion from the cobol copy to my Complex Flat File. The copbol copy book has a structure of one of the field as OCCURS ........
DEPENDING ON ..... INDEXED BY .....
My question is How to define this field? Should it be a GROUP. How to define the range in the Occurs clause. I have seen that we can give only one value in the Occurs clause of the Complex Flat file. Also how to give the Indexd by clause

Please help me in this regard

Thanks
Sorry that my team mate already posted this in DSXchange. Still we didnt get one answer. It would be great if somebody can suggest resolution. As Ray told "OCCURS DEPENDING ON " is not supported. Any other way we can get round this problem
Avishek Mukherjee
Data Integration Architect
Chicago, IL, USA.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I don't know where you found Ray's post stating that the COBOL "OCCURS DEPENDING ON" is not supported; but that is not the case, it is implemented and supported (albeit not well). The Parallel Job Developer's Guide has several examples.

You can also use a hybrid solution by using a server job to process your CFF and write to a sequential file which can then be read by a EE job if you prefer that - the sequential file can even be a pipe to minimize disk I/O.
panic
Participant
Posts: 17
Joined: Fri Sep 23, 2005 5:58 pm

re: solution

Post by panic »

A column that contains an OCCURS DEPENDING ON clause may be flattened
when it is either the last column, or it is followed only by
subordinate columns and none of the following columns contain an
OCCURS DEPENDING ON clause. In all other cases, a column containing
an OCCURS DEPENDING ON clause must be passed as is


so guys this all about occurs depending on in documentation

so how can i implement this particular scenario

05 ICD-4500-PROC-RANGE-CNTR PIC 9(03).
05 ICD-4500-PROCEDURE-RANGES
OCCURS 0 TO 200 TIMES
DEPENDING ON ICD-4500-PROC-RANGE-CNTR
INDEXED BY PROC-RNG-INDEX.
10 ICD-4500-PROC-BEGIN-RANGE PIC X(05).
10 ICD-4500-PROC-END-RANGE PIC X(05).

so gurus anyone can give me solution it would be great

its not about flattening i can't see those column 200 times in the output to map
and also in this scenario which column would be occuring 200 times
procedure ranges or PROC-BEGIN-RANGE and PROC-END-RANGE


thanks
panic
Participant
Posts: 17
Joined: Fri Sep 23, 2005 5:58 pm

re;question

Post by panic »

I also have a question that
OCCURS DEPENDING CLAUSE could be implemented in server jobs
does it support if so
whats the approach and also
if i the file has bad data how can i capture as i am using a reject link from the transformer to capture them down to a sequential file but i could'nt
what's the best approach that i could capture them

thanks
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The Complex Flat File stage supports "fixed OCCURS with no DEPENDING ON clause".
Technical Bulletin Part No. 000-6921 Ascential DataStage Complex Flat File Stage (CFFStage.pdf installed with version 7.5x2, page 2-3)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The document is for the CFF in DataStage Server, in this case it is the PX/EE Version and, according to the documentation, the variable OCCURS DEPENDING ON is supported. If you search the Parallel Job Developer's Guide page the keywords and the "ODO" short form you will see some examples.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

<observation>There's nothing in the document that says "server".</observation>
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Lawrence
Premium Member
Premium Member
Posts: 46
Joined: Thu Jun 10, 2004 12:47 am
Location: Australia

Post by Lawrence »

Hi,
Just went through the same pain of trying to figure out if Datastage server could handle occurs depending on(ODO)clauses - we were looking at Adabas data. We got Ascential support involved and the conclusion from Datastage support was
- Datastage server can handle ODO, however you need to create a seperate output stream from your Complex Flat file stage for each number of occurences . i.e. If your ODO repeats 1 to 3 times then you would have 3 output links from your stage, 1 with the columns defined for all non repeating fields plus the repeating group columns repeated once, the 2nd with the columns defined for all non repeating fields plus the repeating group columns repeated twice and so on. For our purposes this wasn't feasible as we had multiple(up to 24) ODO clauses in a single file and each one could occur up to 191 times. From your description I'm not sure if it would be feasible either as you would need 200 output streams
- Enterpise Edition can handle ODO quite nicely though. We did a proof of concept using EE trying to convince the client to buy EE and it worked quite well(the job, not convincing the client). Using EE basically in a sequential file stage you define each ODO as a variable vector and each column within the repeating group as a subrecord of that vector. The orch defintion looks as follows

For your ODO group

Code: Select all

05 ICD-4500-PROC-RANGE-CNTR PIC 9(03).
05 ICD-4500-PROCEDURE-RANGES
OCCURS 0 TO 200 TIMES
DEPENDING ON ICD-4500-PROC-RANGE-CNTR
INDEXED BY PROC-RNG-INDEX.
10 ICD-4500-PROC-BEGIN-RANGE PIC X(05).
10 ICD-4500-PROC-END-RANGE PIC X(05). 
the Parallel metadata would look like

Code: Select all

  ICD_4500_PROC_RANGE_CNTR:int8 {link_keep};
  ICD_4500_PROCEDURE_RANGES[]:subrec {reference='ICD_4500_PROC_RANGE_CNTR'} (
    ICD_4500_PROC_BEGIN_RANGE:string[5];
    ICD-4500-PROC-END_RANGE:string[5];  
);
To output the columns within the ODO you would need to use a promote sub record stage

Cheers,
Lawrence
panic
Participant
Posts: 17
Joined: Fri Sep 23, 2005 5:58 pm

re: great

Post by panic »

thanks lawrence i really appreciate you
Post Reply