Splitting the file

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
dodda
Premium Member
Premium Member
Posts: 244
Joined: Tue May 29, 2007 11:31 am

Splitting the file

Post by dodda »

Hello

I have a requirement where i will have fixed width flat file with multiple invoices. Each invoice can be indentified by its First line's second character , which is "H" and sixth character would be " " (space) and each invoices last line's second character would be "T" and sixth character would be "M".

The input file can have any number of invoices and i need to produce multiple files such that each file should not have more than 2 invoices.
Suppose if i have 4 invoices (in the example) i need to produce 2 files with 2 invoicdes each.

also each invoice can have multiple transcations

Here is the sample data.
4H000 000 000
4H000A000 000
4L003M002 000
4L004 000 000
4L004M001 000
4L004M002 000
4T999 000 000
4T999M001 001
4T999M001 002
4H000 000 000
4H000A000 000
4L003M002 000
4L004 000 000
4L004 000 000
4L004M001 000
4L004M002 000
4T999 000 000
4T999M001 000
4H000 000 000
4H000A000 000
4L003M002 000
4L004 000 000
4L004M001 000
4L004M002 000
4T999 000 000
4T999M001 000
4H000 000 000
4H000A000 000
4L003M002 000
4L004 000 000
4L004M001 000
4L004M002 000
4T999 000 000
4T999M001 000

Appreciate your help

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

What have you tried? And for the example that you posted, can you please also post the expected output, showing us what each file would need to look like after all the shenanigans?
-craig

"You can never have too many knives" -- Logan Nine Fingers
dodda
Premium Member
Premium Member
Posts: 244
Joined: Tue May 29, 2007 11:31 am

Post by dodda »

Hello Chulett,

thanks for the reply. In the previous post i have given sample data for 4 invoices.

So the out put should be 2 out files with each file having 2 invoices (as we want to have 2 invoices for file)

After splitting the first file should have 2 invoices

4H000 000 000
4H000A000 000
4L003M002 000
4L004 000 000
4L004M001 000
4L004M002 000
4T999 000 000
4T999M001 001
4T999M001 002
4H000 000 000
4H000A000 000
4L003M002 000
4L004 000 000
4L004 000 000
4L004M001 000
4L004M002 000
4T999 000 000
4T999M001 000

second file should have the remaining 2 invoices

4H000 000 000
4H000A000 000
4L003M002 000
4L004 000 000
4L004M001 000
4L004M002 000
4T999 000 000
4T999M001 000
4H000 000 000
4H000A000 000
4L003M002 000
4L004 000 000
4L004M001 000
4L004M002 000
4T999 000 000
4T999M001 000

Regards
Pagadrai
Participant
Posts: 111
Joined: Fri Dec 31, 2004 1:16 am
Location: Chennai

Re: Splitting the file

Post by Pagadrai »

Hi,
The logic to identify two different invoices is not clear.
I hope there is a specific naming convention for the output files.

Anyway, once you identify records for invoices, you can split them into sub files as follows:

In first job, you can process all the records and in the output file, you can add one more column as 'Target File Name'. A transformer stage with proper stage variables will help you.

After the job finishes, you can split the records into multiple sub files based
on the Target file name value. This can be done using a unix script or a server routine and you can call this in a sequence or using after job subroutine.

Let me know if this helps.
Post Reply