multiple record structure flat file

djoni · Post by **djoni** » Wed Jan 18, 2006 3:18 pm

I have a file (one file) extracted from mainframe that has more than one record structures. One record structure is for policy records, the second record structure is for vehicle records, third for driver records, etc. (I work in an insurance company)
The policy record structure has a completely different fields from the others'.
Every record structure has a field, whose value identifies its record structure. For example, if this field value is P, the record is a policy record.
Some fields have the mainframe/COBOL format, e.g. COMP, OCCURS, REDEFINE.
I need to read and process this file in a parallel job.

Can CFF stage handles this file? How?

djoni

ray.wurlod · Post by **ray.wurlod** » Wed Jan 18, 2006 3:39 pm

Have you read the manual for the CFF stage? It's a fairly comprehensive description.

vmcburney · Post by **vmcburney** » Wed Jan 18, 2006 4:44 pm

You need a data dictionary that describes the file, such as a cobol definition file. You import it through manager to create your complex flat file hierarchy, you may need to jiggle it a bit to get it in, for example changing the level numbers or removing certain statements that the importer trips up on.

djoni · Post by **djoni** » Thu Jan 19, 2006 8:51 am

ray.wurlod wrote:Have you read the manual for the CFF stage? It's a fairly comprehensive description.

You bet. I can't find the answer.
djoni

kumar_s · Post by **kumar_s** » Thu Jan 19, 2006 9:09 am

Hi,

All the COBAL/Mainframe file which involves COMP, OCCURS, REDEFINE can be handled by CFF stage.

-kumar

bsreenu · Post by **bsreenu** » Thu Jan 19, 2006 10:01 am

Kumar, Have you ever used CFF stage in PX job? Would it be possible for you to share a sample job? Because I have tried a lot with just simple data and I could not make it work. Any help would be greatly appreciated.

My emailID is bsreenu@yahoo.com, please send me if you have any sample jobs.

djoni · Post by **djoni** » Thu Jan 19, 2006 12:36 pm

kumar_s wrote:Hi,

All the COBAL/Mainframe file which involves COMP, OCCURS, REDEFINE can be handled by CFF stage.

-kumar

I know. Have you used CFF on ONE flat file with MULTIPLE RECORD STRUCTURES? Send me the job. My email is darmawd@thedominion.ca

djoni

sankarsadasivan · Post by **sankarsadasivan** » Mon Apr 09, 2007 11:59 am

We have almost similar requirement and we are planning to use Multi format flat file stage. Is it okay??

Also I have a requirement to check record count and compare it with the trailer record, and abort if theres a mismatch.

I am new to mainframe jobs, I can imagine using wc -l in unix and compare or there are quite many ways to implement in sever/parallel jobs in unix/windows.

Any idea how to do this is mainframe job.

Please help.

ray.wurlod · Post by **ray.wurlod** » Mon Apr 09, 2007 2:59 pm

Mainframe jobs and parallel jobs use a completely disjoint set of stages. There is no comparison. Unless you're running USS, you do not have UNIX commands available for checking files.

Record count within DataStage can be easily achieved by aggregating a single column whose value has been set to the constant 1.

sankarsadasivan · Post by **sankarsadasivan** » Tue Apr 10, 2007 1:44 pm

Thanks Ray.

Yes, also we can use DSGetLinkInfo I believe.
But these techniques, require the whole file to be processed once , to get the record count.

I have to get the counts first, if it doesnt match, I need to error out and not process the file.

Is there any way to achieve this...

Yes, this is more of a mainframe related query..
But just in case, anybody here has done something similar before, in mainframe jobs...

Many thanks,
Sankar

ray.wurlod · Post by **ray.wurlod** » Tue Apr 10, 2007 1:55 pm

Because the mainframe environment does not have the same "command line" concept as UNIX, your best approach is to pre-process the file with a very simple DataStage job that counts the rows (as described earlier). Park the row count somewhere (a file, probably) that you can read to make the decision about whether to proceed, having also read the trailer record.

sankarsadasivan · Post by **sankarsadasivan** » Tue Apr 10, 2007 2:43 pm

Thanks again Ray.

Yes, this seems to be the easy , usual approach.
But again, once I get the counts - processed and trailer counts,

I dont have a Job Control here, all I have is only the Canvas.

How do I do the decission making, where I can compare,
and call child job which actually does the record processing
or make the job fail!

Mainframe canvas doesnt have a sequencer too...

ray.wurlod · Post by **ray.wurlod** » Tue Apr 10, 2007 7:12 pm

You are right - mainframe jobs are restricted to what you can do on the mainframe using COBOL. Therefore, any decision making will need to be performed using DataStage components such as an External Routine stage.

pajj · Post by **pajj** » Thu Apr 12, 2007 7:17 am

We have had success with similar situations doing the following:

1. read the sequential file using a single column string format.
2. in a transformer stage test the control character and route the records to a a distnct path that processes that record type. Use a Column Import stage that will parse the string into the appropriate structure for that path
3. process the parsed data as normal.

tulasi · Post by **tulasi** » Tue Aug 16, 2011 8:29 am

Hi
r u able to resolve this issue.
i am also having a similar issue. please help me in this.

DSXchange