merging multiple line as one record -- Seq data file

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
murur
Participant
Posts: 19
Joined: Wed Apr 14, 2004 7:55 am

merging multiple line as one record -- Seq data file

Post by murur »

I am getting an input data file in the following format

{1111111111111111
:22222
:3333333
:44444444
:55
:66
-}
{
...................
-}

Need to transform into (merge multiple line into ONE record.. number of
lines has to be merged is always constant 7 , first line { and 7th line -}
{1111111111111111:22222:3333333:44444444:55:66-}
{ ...................-}


How do I implement this, Any idea...?

Thank you,
Muru
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Read the data from the file as a single column. Use stage variables to queue data and output on change of group --> the "}" character. Since your last row should be that } character you shouldn't have the issue of the last group not being output.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
rasi
Participant
Posts: 464
Joined: Fri Oct 25, 2002 1:33 am
Location: Australia, Sydney

Post by rasi »

Hi Muru

As Ken said you can do that or you can use the mod function(mod(@INROWNUM,7) to split the records.

Another way to do is create 7 different hash file and merge is one job. While creating the hash file you need to generate the key either by using the stage variable or use the mod function. This is just an another way of doing. But the first mentioned one is the most efficient way of doing.

Regards
Rasi
longma98
Participant
Posts: 9
Joined: Thu Aug 14, 2003 5:12 pm

Post by longma98 »

You can try to preprocess it if it is a unix flat file using sed.

Here is one I stole and modified. Save it in a file and do:

sed -f mySedFile input.dat > output.dat

Basicly, it searches and loops for lines that are not ended with "}" + possible trailing spaces, and removes the "\n" characater.

======
:loop
/} *$/ !{
N
s/\n//
b loop
}
======
Post Reply