How to skip the trailer record

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
crouse
Charter Member
Charter Member
Posts: 204
Joined: Sun Oct 05, 2003 12:59 pm
Contact:
Contact crouse

Post by crouse »

The first byte HAS to be a discrete column. It tells you if the row is a heaer, detail or trailer. Detail records must have a unique designation (not H or T), like "D"? (I'll guarentee you it is defined in the original COBOL copybook.)

If that first byte is not a specific column, you need to read the docs to see how to create a specific column for it.

Then on the "Selection Criteria", check that column for the value "D"... then you'll get just the detail records. That's one of the reasons the CFF stage was created.

Best solution is to have the actual COBOL copybook. That's another reason the CFF stage was created: to decipher the COBOL copybooks. If you are dealing with PACKED data and trying to create the file definition on your own (without the original copybook) you are in for a lot of frustration.

Make every effort to get the original copybook. It exists somewhere.
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Top
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Hi,
This is how the lay out is structured.
Do you think it can be achievable?????
Or do u have any idea how to restructure it????

Code: Select all

      
       01  HEADER-REC-MDBIM.
           05  HEADER-ID-MDBIM               PIC X(1).
           05  HEADER-FILEID-MDBIM           PIC X(8).
           05  SYS-DATE-MDBIM                PIC 9(08).
           05  FILLER                        PIC X(983)  VALUE SPACES.

       01  DETAIL-REC-MDBIM.
           05  PRIMARY-KEY-MDBIM.
               10  COMPANY-MDBIM             PIC 9(02).
               10  LOAN-MDBIM                PIC 9(08).
IAS26A**   05  PRIN-OS-IMP-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  INT-IMP-IAS-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  FEE-IMP-IAS-MDBIM             PIC S9(11)V99      COMP-3.
           05  IMP-MODE-MDBIM                PIC X.
           05  IMP-FLAG-MDBIM                PIC X.
           05  DRP-FLAG-MDBIM                PIC X.
IAS26A     05  FIRST-CHARGE-MDBIM            PIC X.
IAS26A     05  AUTO-LOAN-FLAG-MDBIM          PIC X.
IAS26A     05  REPO-FLAG-MDBIM               PIC X.
IAS26A     05  DISCOUNT-FLAG-MDBIM           PIC X.
IAS26A     05  FEE-COST-EFF-DATE-MDBIM       PIC S9(08)         COMP-3.
IAS26A     05  ORIG-FEE-AMT-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  IRR-INC-COST-MDBIM            PIC S9(03)V9(06)   COMP-3.
IAS26A     05  IRR-EXC-COST-MDBIM            PIC S9(03)V9(06)   COMP-3.
IAS26A     05  ACT-LIFE-MDBIM                PIC 9(03).P-3.
			.
			.
			.
			.
			.
IAS26A***  05  INT-CHG-OFF-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  FEE-CHG-OFF-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A***  05  NON-REV-INT-CHG-OFF-MDBIM     PIC S9(11)V99      COMP-3.
IAS26A**   05  NON-REV-FEE-CHG-OFF-MDBIM     PIC S9(11)V99      COMP-3.
IAS26A***  05  DISCNT-CHG-OFF-MDBIM          PIC S9(11)V99      COMP-3.
IAS26A**   05  IRR-RATE-WOUT-COST-MDBIM      PIC S9(03)V9(06)   COMP-3.
           05  DATE-OF-PETITION-MDBIM        PIC S9(08)         COMP-3.
           05  DATE-OF-ORDER-MDBIM           PIC S9(08)         COMP-3.
           05  DATE-OF-WITHDRAWAL-MDBIM      PIC S9(08)         COMP-3.
IAS26A     05  GRP-RELATED-LOAN-MDBIM.
IAS26A         10 RELATED-LOAN-MDBIM         PIC 9(08) OCCURS 9 TIMES.
IAS26A     05  RELATED-FSV-ALLOC-MDBIM       PIC S9(11)V99      COMP-3.
IAS26A     05  PROV-AMT-LATE-CHG-MDBIM       PIC S9(11)V99      COMP-3.
IAS26A     05  PROV-AMT-INS-PREM-MDBIM       PIC S9(11)V99      COMP-3.
IAS26B     05  FILLER                        PIC X(503)  VALUE SPACES.
IAS26B*    05  FILLER                        PIC X(808)  VALUE SPACES.

       01  TRAILER-REC-MDBIM.
           05  TRAILER-ID-MDBIM              PIC X(1).
           05  RECORD-COUNT-MDBIM            PIC 9(07).
           05  FILLER                        PIC X(992)  VALUE SPACES.

regards
kumar
Top
StephenH
Participant
Posts: 1
Joined: Thu Jun 02, 2005 5:41 am
Location: Cheshire, UK

Post by StephenH »

You can strip out the last part of the file using dd on the unix command line, something like
dd if=<input_file> of=<output_file> bs=1 count=<expected_length>
Assuming there are no delimiters or line terminators, I calculate the trailer length to be 1000 bytes, so expected_length would be the length of input_file minus 1000.

Stephen
Top
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The point is, however, there's no need to do that. Almost everything the CFF stage needs to do that automagically is in the copybook, it just needs to be properly imported. Then I'm assuming that first field - HEADER-ID-MDBIM - can be leveraged to differentiate between header, detail and footer records.

I'll leave the gory details for the other Craig, as this is outside my area of expertise. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Top
crouse
Charter Member
Charter Member
Posts: 204
Joined: Sun Oct 05, 2003 12:59 pm
Contact:
Contact crouse

Post by crouse »

Step aside Mr Hulett, this is a job for ... (I'm sure you'll have fun ending that phrase for me)

Kumar,

I'll take this copybook and "fix it up" for you. In essence, you have 3 distinct copybooks that need to be one (with redefines)

I'll post it some time later today.

Not scolding, but if you're going to live with mainframes and their data, you need know how to play. And a lot of it just comes with experience. And the CFF stage will save a TON of time when dealing with mainframe data, trust me.

-Craig
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Top
crouse
Charter Member
Charter Member
Posts: 204
Joined: Sun Oct 05, 2003 12:59 pm
Contact:
Contact crouse

Post by crouse »

OK, I'll admit when I'm wrong... there is no discrete column the indicates a detail record. Tell your COBOL programmers "thanks".

I'll post a new copybook with REDEFINES, then read the file with the CFF and send to a transformer. In the transformer include a constraint on the output link like this: HEADER-ID-MDBIM <> 'H' and TRAILER-ID-MDBIM <> 'T'

I see you didn't post the complete detail copybook.

Can you post the entire detail copybook?

-Craig
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Top
crouse
Charter Member
Charter Member
Posts: 204
Joined: Sun Oct 05, 2003 12:59 pm
Contact:
Contact crouse

Post by crouse »

Try this with your copy book...

Replace:

Code: Select all

       01  DETAIL-REC-MDBIM. 
           05  PRIMARY-KEY-MDBIM. 
               10  COMPANY-MDBIM             PIC 9(02). 
               10  LOAN-MDBIM                PIC 9(08).
With this:

Code: Select all

       01  DETAIL-REC-MDBIM. 
           05  FIRST_TEN_BYTES.
               10 RECORD_TYPE                PIC X(1).
               10 FILLER                     PIC X(9)
           05  PRIMARY-KEY-MDBIM REDEFINES FIRST_TEN_BYTES. 
               10  COMPANY-MDBIM             PIC 9(02). 
               10  LOAN-MDBIM                PIC 9(08).

Reimport the copybook, import into CFF stage.

Include all columns in the destination.

Out to a transformer stage with a constraint on the output link of:
linkname.RECORD_TYPE <> 'H' and linkname.RECORD_TYPE <> 'T'

That should work for you.

-Craig
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Top
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Hi craig,

Thanks a lot
Thanks for all your effort.
This is really a marvalous idea.
But the layout length for the header detial differs so now i should tell my cobol programmer "THANKS". You can find this in the full layout which is attached below. The filler for the detial is more than the header and trailer.
So the idea you gave again cannot capture the header and trailer record.
I need to find some other way to exclude this else i should ask to provide the data to match the layout.

Code: Select all

        01  HEADER-REC-MDBIM.
           05  HEADER-ID-MDBIM               PIC X(1).
           05  HEADER-FILEID-MDBIM           PIC X(8).
           05  SYS-DATE-MDBIM                PIC 9(08).
           05  FILLER                        PIC X(983)  VALUE SPACES.

       01  DETAIL-REC-MDBIM.
           05  PRIMARY-KEY-MDBIM.
               10  COMPANY-MDBIM             PIC 9(02).
               10  LOAN-MDBIM                PIC 9(08).
IAS26A**   05  PRIN-OS-IMP-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  INT-IMP-IAS-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  FEE-IMP-IAS-MDBIM             PIC S9(11)V99      COMP-3.
           05  IMP-MODE-MDBIM                PIC X.
           05  IMP-FLAG-MDBIM                PIC X.
           05  DRP-FLAG-MDBIM                PIC X.
IAS26A     05  FIRST-CHARGE-MDBIM            PIC X.
IAS26A     05  AUTO-LOAN-FLAG-MDBIM          PIC X.
IAS26A     05  REPO-FLAG-MDBIM               PIC X.
IAS26A     05  DISCOUNT-FLAG-MDBIM           PIC X.
IAS26A     05  FEE-COST-EFF-DATE-MDBIM       PIC S9(08)         COMP-3.
IAS26A     05  IMPAIRMENT-DATE-MDBIM         PIC S9(08)         COMP-3.
IAS26A     05  DISC-PROV-START-DATE-MDBIM    PIC S9(08)         COMP-3.
IAS26A     05  NEXT-UNWINDING-DATE-MDBIM     PIC S9(08)         COMP-3.
IAS26A     05  REPOSESSION-DATE-MDBIM        PIC S9(08)         COMP-3.
IAS26A     05  ORIG-FEE-AMT-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  IRR-INC-COST-MDBIM            PIC S9(03)V9(06)   COMP-3.
IAS26A     05  IRR-EXC-COST-MDBIM            PIC S9(03)V9(06)   COMP-3.
IAS26A     05  ACT-LIFE-MDBIM                PIC 9(03).
IAS26A     05  REM-ACT-LIFE-MDBIM            PIC 9(03).
IAS26A     05  AMORT-DATE-MDBIM              PIC S9(08)         COMP-3.
IAS26A     05  UNAMORT-AMT-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A     05  AMORT-AMT-MDBIM               PIC S9(11)V99      COMP-3.
IAS26A     05  LST-AMORT-AMT-MDBIM           PIC S9(11)V99      COMP-3.
IAS26A     05  CUST-INT-ACC-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  EY-EXC-COST-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A     05  EY-EXC-COST-ACC-MDBIM         PIC S9(11)V99      COMP-3.
IAS26A     05  LST-EY-ADJ-MDBIM              PIC S9(11)V99      COMP-3.
IAS26A     05  FSV-FOR-PROV-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  ALLOC-FSV-PROV-MDBIM          PIC S9(11)V99      COMP-3.
IAS26A     05  DISCNT-FSV-MDBIM          PIC S9(11)V99      COMP-3.
IAS26A     05  DISCNT-RATE-MDBIM       PIC S9(03)V9(6)    COMP-3.
IAS26A     05  DISCNT-PD-MDBIM          PIC 9(03).
IAS26A     05  REM-DISCNT-PD-MDBIM      PIC 9(03).
IAS26A**   05  NON-REV-INT-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  NON-REV-FEE-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  SELL-PRICE-MDBIM              PIC S9(11)V99      COMP-3.
IAS26A**   05  MARKET-VAL-MDBIM              PIC S9(11)V99      COMP-3.
IAS26A**   05  FSV-AMT-AMT-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  FSV-PROV-AMT-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A**   05  INT-PROV-AMT-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A**   05  FEE-PROV-AMT-MDBIM            PIC S9(11)V99      COMP-3.
           05  DISCNT-PROV-AMT-MDBIM         PIC S9(11)V99      COMP-3.
           05  DISCNT-PROV-UNWIND-MDBIM      PIC S9(11)V99      COMP-3.
IAS26A     05  RELATED-PDT-PRIN-MDBIM        PIC S9(11)V99      COMP-3.
IAS26A     05  RELATED-PDT-INT-MDBIM         PIC S9(11)V99      COMP-3.
IAS26A     05  RELATED-PDT-FEE-MDBIM         PIC S9(11)V99      COMP-3.
IAS26A     05  REV-PRIN-MDBIM                PIC S9(11)V99      COMP-3.
IAS26A     05  REV-INT-MDBIM                 PIC S9(11)V99      COMP-3.
IAS26A     05  REV-FEE-MDBIM                 PIC S9(11)V99      COMP-3.
IAS26A     05  REV-LATE-CHG-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  REV-INS-PREM-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  EY-ADJ-MDBIM                  PIC S9(11)V99      COMP-3.
IAS26A     05  NON-REV-INT-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A     05  NON-REV-FEE-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A     05  NON-REV-LATE-CHG-MDBIM        PIC S9(11)V99      COMP-3.
IAS26A     05  NON-REV-EY-ADJ-MDBIM          PIC S9(11)V99      COMP-3.
IAS26A     05  PROV-AMT-PRIN-MDBIM           PIC S9(11)V99      COMP-3.
IAS26A     05  PROV-AMT-INT-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  PROV-AMT-FEE-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  PROV-AMT-EY-ADJ-MDBIM         PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-PRIN-MDBIM            PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-INT-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-FEE-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-LATE-CHG-MDBIM        PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-INS-PREM-MDBIM        PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-EY-ADJ-MDBIM          PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-NREV-INT-MDBIM        PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-NREV-FEE-MDBIM        PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-NREV-LATE-CHG-MDBIM   PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-NREV-EY-ADJ-MDBIM     PIC S9(11)V99      COMP-3.
IAS26A     05  CHG-OFF-DISCNT-PROV-MDBIM     PIC S9(11)V99      COMP-3.
IAS26A     05  TOTAL-SALE-VALUE-MDBIM        PIC S9(11)V99      COMP-3.
IAS26A     05  FNL-SALE-PROCEED-MDBIM        PIC S9(11)V99      COMP-3.
IAS26A     05  FNL-SALE-ALOC-MDBIM           PIC S9(11)V99      COMP-3.
IAS26A     05  TOTAL-CPF-AMT-MDBIM           PIC S9(11)V99      COMP-3.
IAS26A***  05  INT-CHG-OFF-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A**   05  FEE-CHG-OFF-MDBIM             PIC S9(11)V99      COMP-3.
IAS26A***  05  NON-REV-INT-CHG-OFF-MDBIM     PIC S9(11)V99      COMP-3.
IAS26A**   05  NON-REV-FEE-CHG-OFF-MDBIM     PIC S9(11)V99      COMP-3.
IAS26A***  05  DISCNT-CHG-OFF-MDBIM          PIC S9(11)V99      COMP-3.
IAS26A**   05  UN-AMR-FEE-BF-MDBIM           PIC S9(11)V99      COMP-3.
IAS26A***  05  AMR-FEE-WR-OFF-MDBIM          PIC S9(11)V99      COMP-3.
IAS26A**   05  AMR-AMT-MDBIM                 PIC S9(11)V99      COMP-3.
IAS26A***  05  UN-AMR-FEE-CF-MDBIM           PIC S9(11)V99      COMP-3.
IAS26A***  05  IRR-RATE-WITH-COST-MDBIM      PIC S9(03)V9(06)   COMP-3.
IAS26A**   05  IRR-RATE-WOUT-COST-MDBIM      PIC S9(03)V9(06)   COMP-3.
           05  DATE-OF-PETITION-MDBIM        PIC S9(08)         COMP-3.
           05  DATE-OF-ORDER-MDBIM           PIC S9(08)         COMP-3.
           05  DATE-OF-WITHDRAWAL-MDBIM      PIC S9(08)         COMP-3.
IAS26A     05  GRP-RELATED-LOAN-MDBIM.
IAS26A         10 RELATED-LOAN-MDBIM         PIC 9(08) OCCURS 9 TIMES.
IAS26A     05  RELATED-FSV-ALLOC-MDBIM       PIC S9(11)V99      COMP-3.
IAS26A     05  PROV-AMT-LATE-CHG-MDBIM       PIC S9(11)V99      COMP-3.
IAS26A     05  PROV-AMT-INS-PREM-MDBIM       PIC S9(11)V99      COMP-3.
IAS26B     05  FILLER                        PIC X(503)  VALUE SPACES.
IAS26B*    05  FILLER                        PIC X(808)  VALUE SPACES.

       01  TRAILER-REC-MDBIM.
           05  TRAILER-ID-MDBIM              PIC X(1).
           05  RECORD-COUNT-MDBIM            PIC 9(07).
           05  FILLER                        PIC X(992)  VALUE SPACES.
regards
kumar
Top
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:
Contact ray.wurlod

Post by ray.wurlod »

Of course the FILLER is different for different record types - that's what makes the total length the same (1000) in all record types.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Top
Raog
Participant
Posts: 8
Joined: Thu Oct 13, 2005 8:53 am

Re: How to skip the trailer record

Post by Raog »

Hi,

With my limited knowledge on Datastage, I am not sure whether u can use "Tail stage" which they provided unsder development/debug.

Thx,
Rao.
kumar_s wrote:Hi,
How to skip the trailer record which has a different layout from the detail record. File is in ebcidic format. How can we do this with out any warning.

Regards
kumar
Top
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Raog,

Thanks for your suggestion. Tail stage is retrive the tail records and not to skip the tail records.

Regards
Kumar
Top
Post Reply

Return to “IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)”