multiple record structure flat file
Moderators: chulett, rschirm, roy
multiple record structure flat file
I have a file (one file) extracted from mainframe that has more than one record structures. One record structure is for policy records, the second record structure is for vehicle records, third for driver records, etc. (I work in an insurance company)
The policy record structure has a completely different fields from the others'.
Every record structure has a field, whose value identifies its record structure. For example, if this field value is P, the record is a policy record.
Some fields have the mainframe/COBOL format, e.g. COMP, OCCURS, REDEFINE.
I need to read and process this file in a parallel job.
Can CFF stage handles this file? How?
djoni
The policy record structure has a completely different fields from the others'.
Every record structure has a field, whose value identifies its record structure. For example, if this field value is P, the record is a policy record.
Some fields have the mainframe/COBOL format, e.g. COMP, OCCURS, REDEFINE.
I need to read and process this file in a parallel job.
Can CFF stage handles this file? How?
djoni
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
You need a data dictionary that describes the file, such as a cobol definition file. You import it through manager to create your complex flat file hierarchy, you may need to jiggle it a bit to get it in, for example changing the level numbers or removing certain statements that the importer trips up on.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
multiple record structure flat file
Kumar, Have you ever used CFF stage in PX job? Would it be possible for you to share a sample job? Because I have tried a lot with just simple data and I could not make it work. Any help would be greatly appreciated.
My emailID is bsreenu@yahoo.com, please send me if you have any sample jobs.
My emailID is bsreenu@yahoo.com, please send me if you have any sample jobs.
kumar_s wrote:Hi,
All the COBAL/Mainframe file which involves COMP, OCCURS, REDEFINE can be handled by CFF stage.
-kumar
I know. Have you used CFF on ONE flat file with MULTIPLE RECORD STRUCTURES? Send me the job. My email is darmawd@thedominion.ca
djoni
-
- Premium Member
- Posts: 39
- Joined: Tue Dec 23, 2003 3:47 am
- Location: India
We have almost similar requirement and we are planning to use Multi format flat file stage. Is it okay??
Also I have a requirement to check record count and compare it with the trailer record, and abort if theres a mismatch.
I am new to mainframe jobs, I can imagine using wc -l in unix and compare or there are quite many ways to implement in sever/parallel jobs in unix/windows.
Any idea how to do this is mainframe job.
Please help.
Also I have a requirement to check record count and compare it with the trailer record, and abort if theres a mismatch.
I am new to mainframe jobs, I can imagine using wc -l in unix and compare or there are quite many ways to implement in sever/parallel jobs in unix/windows.
Any idea how to do this is mainframe job.
Please help.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Mainframe jobs and parallel jobs use a completely disjoint set of stages. There is no comparison. Unless you're running USS, you do not have UNIX commands available for checking files.
Record count within DataStage can be easily achieved by aggregating a single column whose value has been set to the constant 1.
Record count within DataStage can be easily achieved by aggregating a single column whose value has been set to the constant 1.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 39
- Joined: Tue Dec 23, 2003 3:47 am
- Location: India
Thanks Ray.
Yes, also we can use DSGetLinkInfo I believe.
But these techniques, require the whole file to be processed once , to get the record count.
I have to get the counts first, if it doesnt match, I need to error out and not process the file.
Is there any way to achieve this...
Yes, this is more of a mainframe related query..
But just in case, anybody here has done something similar before, in mainframe jobs...
Many thanks,
Sankar
Yes, also we can use DSGetLinkInfo I believe.
But these techniques, require the whole file to be processed once , to get the record count.
I have to get the counts first, if it doesnt match, I need to error out and not process the file.
Is there any way to achieve this...
Yes, this is more of a mainframe related query..
But just in case, anybody here has done something similar before, in mainframe jobs...
Many thanks,
Sankar
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Because the mainframe environment does not have the same "command line" concept as UNIX, your best approach is to pre-process the file with a very simple DataStage job that counts the rows (as described earlier). Park the row count somewhere (a file, probably) that you can read to make the decision about whether to proceed, having also read the trailer record.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 39
- Joined: Tue Dec 23, 2003 3:47 am
- Location: India
Thanks again Ray.
Yes, this seems to be the easy , usual approach.
But again, once I get the counts - processed and trailer counts,
I dont have a Job Control here, all I have is only the Canvas.
How do I do the decission making, where I can compare,
and call child job which actually does the record processing
or make the job fail!
Mainframe canvas doesnt have a sequencer too...
Yes, this seems to be the easy , usual approach.
But again, once I get the counts - processed and trailer counts,
I dont have a Job Control here, all I have is only the Canvas.
How do I do the decission making, where I can compare,
and call child job which actually does the record processing
or make the job fail!
Mainframe canvas doesnt have a sequencer too...
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You are right - mainframe jobs are restricted to what you can do on the mainframe using COBOL. Therefore, any decision making will need to be performed using DataStage components such as an External Routine stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
We have had success with similar situations doing the following:
1. read the sequential file using a single column string format.
2. in a transformer stage test the control character and route the records to a a distnct path that processes that record type. Use a Column Import stage that will parse the string into the appropriate structure for that path
3. process the parsed data as normal.
1. read the sequential file using a single column string format.
2. in a transformer stage test the control character and route the records to a a distnct path that processes that record type. Use a Column Import stage that will parse the string into the appropriate structure for that path
3. process the parsed data as normal.
Re: multiple record structure flat file
Hi
r u able to resolve this issue.
i am also having a similar issue. please help me in this.
r u able to resolve this issue.
i am also having a similar issue. please help me in this.